TURBOT
title Scrape Bank of the republic of Haiti (bot_mission_769) pending_draft_review
description Scrapy scrape scrape
current run state not running
last run single run snapshot draft scrape succeeded on December 17, 2014 23:33
next run n/a
created by justus ( )
last reviewed by peter.evans
(no subject)
peter.evans commented almost 9 years ago

Hi Justus,
Hope all is well with you. Thank you for the work that you have put in to this scraper so far (scraping the bank of the republic of Haiti (bot_mission_769)).
I thought I would summarise where I believe this bot is in terms of progress in case you would like to continue working on it:
We have a scraper that is producing good output which is compatible with out Turbot framework, I have reviewed the bot and made these comments (I will paste here as they were made in a different thread)
"""
- Firstly, there seems to be some extra data available which could be very good to scrape, namely Established Date, Bank Type, Head Office Address.
- Secondly, a transformer to format the data according to our Simple licence schema would be a really great addition to the primary data. The documentation on our data types is here: http://turbot.opencorporates.com/docs/supported_data_types (minimum Simple licence schema required fields are actually only "source_url", "sample_date", "company_name", "company_jurisdiction"). Also some examples here: http://turbot.opencorporates.com/docs/examples#structured-bots
"""
These still look like good action points to me, but you may have progressed the bot further than this locally already.
Please do be in touch if I can help with anything.
All the best,
Peter

(no subject)
peter.evans commented over 9 years ago

Hi Justus,
No problem, great to hear that you're keen to keep working on the bot.
Thanks for clarifying about the duplicate, I'm sorry that you ran into problems with the first one, I'll delete 'mission_769'.
Let me know if any questions come up, particularly regarding writing a transformer to output a second set of, standardised, data - as that is a really great thing to add.
Thanks again & do feel free to be in touch.
Peter

Re: (turbot bot [bot_mission_769])
commented over 9 years ago

Hello Peter,
thanks for contacting me - the bot ‘mission_769’ was inoperable. When I tried uploading a new version for testing using the turbot tool that you gave me, I was only getting silent crashes. So if you could delete ‘mission_769’, that would be very nice.
As always - thanks!
Justus

(no subject)
peter.evans commented over 9 years ago

Hi Justus,
This bot seems to have appeared twice on the system - if you can let me know which is currently being worked on I can delete the duplicate. The bot names are: bot_mission_769 and mission_769, I believe both pertain to the same dataset.
Thanks!
Peter

Bot state update
commented over 9 years ago

A draft run succeeded; sending for review

Bot state update
commented over 9 years ago

The bot was pushed; scheduling a draft run

Run history

event metadata
single run snapshot draft scrape succeeded on December 17, 2014 23:33 9 rows in less than a minute

Config

{
  "bot_id": "bot_mission_769",
  "title": "Scrape Bank of the republic of Haiti",
  "description": "Scrapy scrape scrape",
  "language": "python",
  "data_type": "primary data",
  "identifying_fields": [
    "number"
  ],
  "company_fields": {
    "name": "company"
  },
  "files": [
    "scraper.py"
  ],
  "frequency": "monthly",
  "publisher": {
    "name": "Publisher of the data",
    "url": "Publisher's website",
    "terms": "Copyright terms (e.g. Open Government License, n/a, etc)",
    "terms_url": "A place where these terms can be checked or verified"
  }
}