12

Reference : Extracting tables from multiple PDFs

Some close vote raised the question of techniques and particularly data extraction techniques as on- or off-topic. One can argue some other sites, such as SO, already have such questions.

I would say they are on-topic, because they are useful to Open Data users, but also, more simply, because it is part of the "Open" in Open Data. Indeed, better extraction techniques make for more open data.

Should data extraction techniques be on- or off-topic?

14

Dealing with large collections of PDFs is a common problem among programmers, researchers, etc., in the open data/open government space, and being able to poll this particular community as opposed to the broader programming community is useful since we're likely to have dealt with very similar circumstances in the past. Further, the solution to this problem may not end up involving code -- perhaps off-the-shelf end-user tools end up doing what's needed.

Also, if pulling structured data out of a text file with Refine is on-topic, I'm not sure why pulling structured data out of a PDF with a similar tool wouldn't be.

  • 1
    It may also help developers identify tools that could be particularly helpful to a coherent group of people sharing the same interest. – Roger_S May 21 '13 at 21:05
5

I agree that data extraction techniques should be considered on-topic for this site.

  • People working in this space encounter similar problems, and some of them may be particular to the type of data put out by government systems.
  • Data extraction questions for a particular dataset could lead to others pointing an OP to an already-extracted or better source for the data. Such an interaction would be unlikely to happen on a more general Stack Exchange.
4

The data extraction technique is more than just programming.

In the cited question about PDF extraction, the programming component is dominating and it could be formulated "How to extract tables from PDF in [programming language]" on StackOverflow.

But in general, to extract the data from given source, it is often necessary to do some revert engineering about the structure and the interpretation of the data - especially if the data is given not in relational database dump, but in some CSV or must be scrambled.

Such questions are the best to be opened here, on SO they would be definitely off-topic.

1

I agree with Andrew that code is not the only possible answer. I like to add that there could be non-programming components in the ultimate solution, for example, how to verify the results.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .