At the OECD my team maintains many Excel webqueries to collect data from country web sites rather than the country using resources to package the data and send it. Each Excel webquery represent a separate dataflow (country+subject domain) and we have over 200 of these dataflows.
Excel webqueries offer a powerful interface and processing language on the results (VBA). However, this whole system is difficult to maintain because:
- Each Excel web query is a separate code base (therefore many different code bases)
- The webquery definitions cannot be searched, batch-edited or inherited
- The result processing can't be seperated from the webquery definition
- The webquery execution can't easily be placed on a server to be scheduled
- It is difficult to add Excel webqueries in a server-based workflow
Does anyone have or know a suitable software alternative to Excel webqueries that solves at least some of these issues?
David Barraclough, OECD