4 articles Documentation

Reference docs for core features like Arguments, Query, Dynamic URL and Callback

Dynamic URL and headers

Using this feature of dynamic URL and headers, you can dynamically create a source URL and scrap content on the fly. Basically this lets you fetch content from a single underlying source by passing multiple get arguments to it. This feature will convert specific text in your URL or headers arguments to corresponding value based on some placeholders. For…

Query

This article specifically details usage of query which are the heart of WP Web Scraper. For parsing html, the plugin three types of queries- CSS Selectors; XPath and Regular Expression. Selectors are not only used by WP Web Scraper to query data from source URL, but also to remove or replace stuff For all scraping that deals with DOM…

Arguments API

Unless mentioned, all these arguments are available in the template tag as well as shortcode. Here’s how these arguments will be used. In shortcode: Error parsing: Query returned empty response In template tag: <?php echo wpws_get_content(“https://www.yahoo.com/”, “ol.trendingnow_trend-list”, array(‘output’ => ‘text’)); ?> For representational purposes, these arguments are categorized below as Request, Response or Parsing ones. Request…