How to get (scrape) the contents of a site that requires logging in through YQL?
Is it possible to get (scrape) data fr开发者_运维问答om a site that requires logging in using YQL? If yes, please tell the procedure.
You'll need the user to authorize your access via OAuth, as YQL's docs mention. In addition to the docs pointed to by links from the URL I just mentioned, you can learn all about OAuth here, then get libraries to help you use OAuth, depending of course on the programming language you want to use, from the links listed here.
Depending on how the remote site is set up, you could use a simple POST (there is an open data table for that1) or you could create your own small, custom data table and use <execute>
2 to send whatever headers (including Cookie:
) you need over one or more GET/POST requests.
- htmlpost data table (example)
- YQL Execute
精彩评论