Fetching plain text in Yahoo Pipes
I have a Yahoo pipe taking the Atom feed from a Google group, and I want to do some processing on the message's full text (running various regular expressions to extract data). I can get a message's text in plain text from from Google using a url like this:
http://groups.google.com/group/(group_name)/msg/(message_id)?dmode=source&output=gplain
However, I'm having trouble getting it inside Yahoo pipes as a string value. Fetch Page rejects non-HTML pages. YQL using the html table seems to work, and wraps the plain text inside a p element, whose text I can extract like this:
select * from html where url="..." and xpath="//p"
However, if the message text contains html tags, YQL returns an HTML subtree instead of a string. Is there any way of flatt开发者_高级运维ening it back into its HTML source?
The trick is to remove the "output=gplain" and grab the content from the pre element.
select content from html
where url="http://groups.google.com/group/haml/msg/0f78eda2f5ef802d?dmode=source"
and xpath='//div[contains(@class,"maincontbox")]/pre'
I have created a pipe with Google Group and Message ID as inputs to demonstrate:
http://pipes.yahoo.com/pipes/pipe.info?_id=3d345e162405e7dbd47d73b95c21f102
精彩评论