Three weeks ago, Yahoo announced a real impressive tool for developing web mashups called Yahoo Pipes. In the UNIX and command line world, a pipe “|” represents a joining of two programs, taking the output of one and making it the input for another, until you finally get the result you want. i.e. cat log.txt | grep “ERROR”. Yahoo Pipes is really just a conduit for data between web services, much the same as in UNIX where a pipe is a conduit for the transfer of data between programs. I can’t help but think of the wonderful possibilities this web app could have. As Tim O’Reilly states in his blog entry about Pipes “This is something I’ve been waiting nearly ten years for.” A bold and important statement coming from the man who coined the phrase “Web 2.0”.
Now, the concept of Pipes is not new. UNIX has it. Even Apple has a similar type of desktop application called Automator. What is new is that it is all done using a web browser AJAXified interface. You do not need a lot of web programming experience to get started, as you can simply take existing modules and mash them up into your own Pipe. You can then share your Pipe with others and you’ll quickly get the concept of code reuse.
The Pipes Editor – URL Builder allows you to fetch any data source via its RSS, Atom or other XML feed, extract the data you want, combine it with data from another source, apply various built-in filters (sort, unique, count, truncate, union, join, as well as user-defined filters), and apply simple programming tools like for loops. You can solicit user input and build URL query strings to submit to REST-based sites. You don’t just get to look at the output of your Pipes program on the Pipes web site. Output options include RSS, JSON, and you can even get results by email or SMS. Popular uses of Pipes is to use it for searching multiple sites at once and getting aggregated results, or to get alerted to items that may interest you. I wrote one example called “Deals On Memory” that searches sites like bensbargains.net and techbargains.com so that I get alerted to the latest deals on computer memory.