Hi Tom,
Thank you for being so thorough with this Mission, the amount of data captured is extremely impressive!
I’ve spent quite a lot of time looking at the extra data captured into primary data and it looks good - we can also have a closer look at that when it comes time to transform various parts of it.
The status reporting that you built in is very useful - it’s flagged up the areas where it looks like we’re not scraping everything: https://turbot.opencorporates.com/bots/ar-licences/runs/2348/metadata
Specifically it looks like it is only scraping the first 8 institutions in receivership. There’s also an ‘unable to load’ debugging message for financial trusts, which I don’t think are being scraped currently, unless I’m missing something. Otherwise it looks like all categories are being scraped.
There is also an ‘unable to load’ message for private banks, public banks, and financial institutions, but they seem to have scraped okay anyway.
The only other thing I can see to point out is that in the manifest we refer to “Argentinean” which I think should be “Argentinian” or even better “Argentine”.
Thanks again for working on this scraper - please let me know if I’ve misinterpreted anything in this review, it’s not unlikely given the complexity of the data :)
Best wishes,
Peter