WordPress.org

Ready to get started?Download WordPress

Forums

WP Web Scraper
Can only be used by experienced coders (2 posts)

  1. tongla
    Member
    Posted 1 year ago #

    Hi
    Sorry to trouble you.

    I downloaded the plugin thinking it could be used with a button as I am very new to web pages etc.

    "Can be easily implemented using the button in the post / page editor."

    Where is the button?

    Tried putting the code in a page as per ;

    "For use directly in posts, pages or sidebar (text widget): [wpws url="" selector=""]"

    My code;

    [wpws url="https://secure.publicworks.qld.gov.au/etender/tender/search/tender-search.do?action=advanced-tender-search-open-tender&orderBy=closeDate" selector="title" user_agent="My Bot" on_error="error_show"]

    I only get this error.

    Error fetching https://secure.publicworks.qld.gov.au/etender/tender/search/tender-search.do?action=advanced-tender-search-open-tender&orderBy=closeDate - Operation timed out after 2000 milliseconds with 0 bytes received

    "Configurable caching of scraped data. Cache timeout in minutes can be defined in minutes for every scrap."

    Where can the timeout be configured?

    How can this be made to work?

    Kind regards

    Tongala

    http://wordpress.org/extend/plugins/wp-web-scrapper/

  2. Akshay Raje
    Member
    Plugin Author

    Posted 1 year ago #

    @tongla I would agree that this is more like a developer tool than an end-user plugin. I have tried to document most that I can in the manual here -http://wordpress.org/extend/plugins/wp-web-scrapper/other_notes/

    With reference to the shortcode you have mentioned, there is nothing wrong - just that the response from that website is slow and the scraper timesout by default in 2 seconds. Make the following change to make it work:

    [wpws url="https://secure.publicworks.qld.gov.au/etender/tender/search/tender-search.do?action=advanced-tender-search-open-tender&orderBy=closeDate" selector="title" timeout="10"]

    For any further help, please write to me at akshay[dot]raje[at]gmail[dot]com

Topic Closed

This topic has been closed to new replies.

About this Plugin

About this Topic