David inquired about your bot in Slack so I responded saying this:
This is a great bot, it was really thorough, lots of information available there. I responded via our turbot mail system (Hope that got through?) outlining the basic idea of a licence transformer.
Scraping the primary data is brilliant but in order to normalise data from different sources and import it into opencorporates we need to apply licence transformers. We are only using the simple-licence transformer for MTB at the moment, we have a much better rich-licence transformer ready but we need to document it before people can apply it easily.
So my answer is that there are no issues with the bot (it is great!) but with a licence transformer it could be even better
I tried to send you some mail with Turbot previously but perhaps there some problem with our system, I'll paste what I said below:
As I said this looks like a really tricky dataset to scrape, well done! I'll give this scraper a proper review soon but it looks like the kind of dataset that merits a very thorough review. Was this one that was started at the Madrid FlashHacks?
In the meantime can I suggest that you have a look at our tutorial examples for how to apply a basic transformer to the dataset? A transformer outputs a second set of data which is formatted such that the data may be ingested into the main OpenCorporates database (& thus made open).
You can find the examples for how to construct a transformer here: http://turbot.opencorporates.com/docs/examples#structured-bots
and the description of the simple licence schema here: http://turbot.opencorporates.com/docs/supported_data_types
If you have any questions do feel free to be in touch by responding to this email - I've also invited you to our Slack group.
All the best & thank you for writing this scraper!
So in summary the bot is in great shape, how do you feel about writing a licence transformer? :)
All the best,