Welcome to shell: revealed Sign in | Join | Help
in Search

Shell Blog

Query Composition: Building a search upon another search

Today, I'd like to share with you a hidden gem inside of Windows Vista's search capabilities.  It's a neat little feature that lets you build one saved search on top of another, and we call it Query Composition.  Think composition in the mathematical sense, not in the context of writing an English paper.  To explain this feature, I'd like to start by talking a bit about Libraries.

If you've followed the development of the search features in Windows Vista, you'll know that we played around with the idea of Libraries for quite a while.  Instead of each user having their special Documents, Music, and Pictures folders on the Start Menu, there were the Documents Library, the Music Library and the Pictures Library.  These libraries were essentially a saved search that ran over all your files and found the appropriate kind for each library. 

What many people didn't realize is that all of these were built on top of the one master library aptly named Library, which was just an unqualified search for all of your files.  Each of the kind-specific libraries were defined like this:  "Find all results in Library that are of kind <Document, Picture, Music, etc.>"  This makes a lot of sense, because you'd expect to find all the documents in your Documents Library in your Library as well, along with a bunch of other kinds of files.  It also had the benefit of automatically updating your Documents, Pictures, Music, etc. libraries anytime you updated Library.  Back around the Beta 1 timeframe, we provided mechanisms for adding new folders to your Library.  By default, Library just searched over your user profile and the Public profile, but in every folder there was a button "Add to Library" that would let you add it to the definition of your Library (when possible, it would also add the folder to the index).  And as soon as you did, all your specialized libraries were automatically updated because they referenced the one Library. 

We originally designed query composition as a way to build powerful searches and save a set of search locations (which we called “scopes”) that could be reused in other searches.  When we created Libraries, we realized that we could easily leverage the query processing engine we had written to build the kind-specific libraries on top of the one master Library.  As development of Windows Vista continued, we got a lot of feedback that said people weren't ready to move to a library-based file explorer just yet, so we scaled back on the Library vision a bit.  But all the pieces to compose a search on another are still there; when we restructured the way we presented saved searches to the end user, we didn't modify our query processing engine.

So, how do you go about composing a search on another?  First, start by saving a useful search.  Perhaps you save a search for Show only:Document, Author=<Your Name>, Size>1KB using the Advanced Search pane, naming it "Documents I Wrote". 

Creating Documents I Wrote

Now you want to make an "Important Documents" search that is all of the documents you wrote that have the Tag "Important".  Instead of re-doing all the work you did to build “Documents I Wrote”, you can just build “Important Documents” by composing on “Documents I Wrote.”  To do this, open the Advanced Search pane and pull down the Location drop-down.  Near the bottom is a Choose Search Locations... entry - select that one.  You should see a dialog like this:

Composing on Documents I Wrote

Now, scroll down to the bottom of the tree at the top and expand the “Search Folders" node.  Check "Documents I Wrote" and then press OK.  Now add your Tags=Important filter in the Advanced Search pane and press Search.  You should see that you now have all the documents you wrote that are tagged Important.

Creating Important Documents 1

The previous example is rather simplistic, but the query processing engine in Windows Vista allows you to do more powerful things like compose upon more than one search at a time.  Start by saving a "Documents Justin Wrote" search with Show only:Document, Author=Justin, Size>4KB

Creating Documents Justin Wrote

Now let's assume you are working on a project where your important documents are written by either you or Justin.  Try building “Important Documents” on top of both “Documents I Wrote” and “Documents Justin Wrote” - in the Choose Search Locations dialog just check both of these search folders.  When you run this search, you'll see that you get documents tagged Important written by either you or Justin!

Creating Important Documents 2

This process easily scales to allow you to select many search folders as the basis of the search you're creating.  You can even select both search folders and regular physical folders as the locations for a search.  Additionally, if the search folder you select when creating a new search happens to be defined as referring to yet another search folder, Windows Vista's query processing engine will handle that properly, too.

Here's another interesting point - if you happen to change a search folder that is referenced by another search, the results will automatically be updated in both searches.  I know that editing searches isn't very easy in the UI that Explorer provides, but we hope to release a Windows Vista Power Toy that will allow you to edit search folders and manipulate significantly more complicated filters.  It should be similar to the advanced search pane that was available in Beta 1 of Windows Vista.

Finally, you may be interested to know that the Library concept hasn’t been completely removed from Windows Vista.  All of the Recent <Documents, Pictures, etc.> searches that are automatically installed in your Searches folder leverage this ability - they're all composed upon a search folder called "Indexed Locations," which automatically updates to search whatever folders you've configured to be indexed.  Indexed Locations is also the default search location for all the searches you run by clicking on the Search link in the Start Menu, and any search you save from there will be composed upon “Indexed Locations” as well.

Published Tuesday, October 31, 2006 2:26 PM by benbetz

Comments

 

CoLD-FiRe said:

Great to hear guys!

As you guys probably read in the forums. The removal of the Beta 1’s search advanced was really disappointing!

But its good to see that there might be a powertoy created for it!

Just a quick suggestion here,

You guys said that people weren’t ready for a Libraries.

I believe we would have been, but you guys didn’t implement it clearly enough.

Let me explain…..

Before the reset in the Longhorn builds. You had the Libraries displayed in My Computer. So in other words you had virtual folder links which showed ALL files of a certain file type e.g. Music, Images etc.

Then after the reset you guys stilled used the Library concept BUT use never did place them back in My Computer again. But instead had them linked straight to the start menu’s shortcuts with NO indication that the links linked to virtual folders and not normal folders. For the user to know the difference they would have had to notice that the folder icon in the address bar was Blue and not Yellow. And if this was a first time user, they wouldn’t have a clue.

Another thing to mention is, In the earlier builds the it actually said Library up in explorers title bar. So you would go, “Hey, I’m in “The Music Library” (which displayed all music indexed) And NOT the My Music folder (which simply is a normal folder).

Then again, after the reset you guys for some reason REMOVED the title bar headings! WHY? I ask WHY?!

So in conclusion, what I’m saying is…

Users WOULD be ready for Libraries RIGHT NOW in Vista.

You simply implemented them in a poor way AFTER the reset.

I beat you, you could put a sub category in Vista’s My Computer called Libraries.

Which would Simple use virtual folders to display ALL MUSIC, ALL PICTURES, ALL DOCUMENTS etc. And you would find there wouldn’t be a person that would have a problem understanding it! As long as you made it clear in Explorers title bar that they were viewing a Library Explorer. And DIDN’T link them straight to the start menus shortcuts!

Think about this for the Next Release!!! You will find I’m correct!

November 1, 2006 12:17 AM
 

cchance said:

hehe i gotta agree a libraries in my computer would have worked fine if clearly designated as "The Music Library" etc... but then again we still dont have a videos link in the start menu or an option to set one... nope just pictures and music, i dont have music i have music VIDEOS :)

November 1, 2006 12:36 AM
 

CoLD-FiRe said:

It appears that not many people care about the how “search and organize” vision.

The reason why I care about it so much is because I have 2.8tbs of data spanned across many drives. And keeping track of it all is a nightmare. Hence why I wanted Vista to make better use of Virtual Folders, Auto/Lists and Tagging.

Vienna maybe?

November 3, 2006 6:12 AM
 

benbetz said:

We will certainly continue to inovate in the search and organize space.  Post-reset, we removed our dependency on WinFS and had to reconsider what we show in the Computer portion of the namespace.  This has traditionally been restricted to data stores like hard drives, attached devices, and remote shares mapped as a persisted connection on the local machine (either via a mapped drive or a "Network Location").  Since the Libraries we were developing were no longer true data stores, but rather a view over other existing data stores, we wanted to place them in a different location in the namespace.  

As you've mentioned, the solution we developed didn't turn out to be the clearest entry point for most users - perhaps I should have said that users weren't ready for Libraries to be their main, default view from the Start Menu.  When we simplified things based on this feedback, there was a lot of other feedback that we received as well.  After analyzing all of it, we determined that we needed to focus on building a clear, easy-to-use UI first, since that's what our users were demanding above all else.  We will continue to look at the Library concept for future releases.  I personally hope we can get to the point where the "yellow" folders are a thing of the past, and the common file browsing experience centers around quick, dynamic searches/stacks/lists.  I don't know how many times I've had to move my music files around a complicated folder structure to organize them by artist, album, etc. - I'm ready to be rid of that for good! We're not there yet, but I think Windows Vista starts us in the right direction.

November 3, 2006 8:45 PM
 

CoLD-FiRe said:

THANK YOU Benbetz for your reply!

One point about the namespace with Libraries and what ms reconsidered,

I can understand you wanting to stick to just having hard drives, removable media and network drives only displayed under my computer. But, in XP this really isn’t the case.

Since we have all that plus, Shortcuts to Shared Documents and Users Documents. And then we also have under the “Other” heading, links to things like Scanners, WLM Sharing Folders, Phone Browsers etc. So I don’t really see how having Libraries listed there would make things confusing.

But anyway, I’m not trying to cause trouble! Just wanted to know what happened to the whole concept. So thanks again Benbetz for replying/explaining!

“I personally hope we can get to the point where the "yellow" folders are a thing of the past, and the common file browsing experience centers around quick, dynamic searches/stacks/lists.”

Oh say no more! I totally want that too! :)

I’m just worried that we won’t see this until the next 2 or more releases of Windows!

P.S. One thing that made Virtual Folders confusing was. When you had say, the Documents VF open for example, you couldn’t Drag and Drop files into it. And you couldn’t save to VF’s. So you would then have to do a further step of opening up the Documents directory just to be able to drop files into it!

Ok now your point would be, Yes, because its NOT a directory so you cant save or drop files into it! But since VF’s were launched by default. You would still think they would at least handle file operations the same was as a real folder does.

I believed a simple way to Solve this problem, would have been for files to be automatically redirected to a specific directory relevant to the file type. For example if a user saved/drag & dropped some Mp3s to the All Music VF the files would be moved to the Music Folder.

If they saved a Word doc to the All Documents VF the doc would be redirected to the Documents folder. This way if the user uses either the VF/Library or manually browses to the directory, they would still find there document/file etc.

November 4, 2006 11:43 AM
 

benbetz said:

If you had a chance to play with some of the Longhorn Alpha builds, back when we had WinFS, you may have noticed that Virtual Folders did allow saving and automatic metadata painting.  Unfortunately, after the reset we had to cut this feature, as it wasn't going to fit into the schedule we had to meet for Windows Vista.  We know that people want it - we want it too!  We'll certainly look at adding it in future releases of Windows.

November 6, 2006 12:57 AM
 

Gadget Corner said:

November 27, 2006 5:35 PM
Anonymous comments are disabled

About benbetz

I have been a developer on the Windows Shell Team at Microsoft since 2004, contributing mostly to the Search and Organize features. Prior to that I attended DePauw University where I did research on Assistive Technology.
Powered by Community Server, by Telligent Systems © 2006 Microsoft Corporation. All rights reserved. Terms of Use | Trademarks | Privacy Statement.