title Financial Licences of the Dominican Republic (do-valores) pending_draft_review
description This bot retrieves information about licensed securities firms in the Dominican Republic
current run state not running
last run single run snapshot draft scrape succeeded on February 04, 2016 10:35
next run n/a
created by dinotash (Tom Curtis)
last reviewed by peter.evans
Bot state update
commented about 8 years ago

Run succeeded; sending for draft review

Bot state update
peter.evans commented about 8 years ago

A failed draft run was restarted by the moderator

Saved vars cleared
dinotash commented about 8 years ago

Bot state update
commented over 8 years ago

Run succeeded; sending for draft review

Bot state update
dinotash commented over 8 years ago

The bot was pushed; scheduling a draft run

Bot state update
commented over 8 years ago

A draft run failed

Bot state update
dinotash commented over 8 years ago

The bot was pushed; scheduling a draft run

Bot state update
dinotash commented over 8 years ago

The bot was pushed; scheduling a draft run

(no subject)
peter.evans commented over 8 years ago

Hi Tom,
Thanks for pushing this again, have responded on Slack but will add the specific info here:
I'm looking at do-valores right now - this is looking really good - it's testing my knowledge of the licence schema though! I think shareholders should not be transformed as officers as they're to do with ownership rather than official representation. We have two options here for do-valores - if you have time we can make an edit to include the shareholders + their share parcels, if not we should just exclude them from the officers field (& we can add them in properly later)
the fix to include them as shareholders I believe is to switch the entity type from "unknown_entity_type" to "company-for-nesting" - this schema then has a "share_parcels" attribute (https://github.com/openc/openc-schema/blob/master/schemas/includes/company-for-nesting.json#L83) which can take an array of existing share parcels as per this schema: https://github.com/openc/openc-schema/blob/master/schemas/includes/share-parcel.json using which we should be able to record the share parcels + shareholders.
So the format will be something like:
:share_parcels =>
[ { :number_of_shares => 1234,
:shareholders => [ {:name => 'Fred Flintstone'} ],
:confidence => 42
},
{ :percentage_of_shares => 23.5,
:shareholders => [ {:name => 'Barney Rubble'},
{:name => 'Wilma Flintstone'} ]
},
]
},
In this case I think we have number of shares and percentage of shares.
Let me know if this is unclear and I can make up a better worked example.
Thank you,
Peter

Bot state update
commented over 8 years ago

Run succeeded; sending for draft review

Bot state update
dinotash commented over 8 years ago

The bot was pushed; scheduling a draft run

Bot state update
commented over 8 years ago

A draft run failed

Bot state update
dinotash commented over 8 years ago

The bot was pushed; scheduling a draft run

Bot state update
commented over 8 years ago

A draft run failed

Bot state update
dinotash commented over 8 years ago

The bot was pushed; scheduling a draft run

Run history

event metadata
single run snapshot draft scrape failed on August 30, 2015 15:25 86 rows in about 4 hours
single run snapshot draft scrape failed on August 30, 2015 22:57 86 rows in about 4 hours
single run snapshot draft scrape succeeded on August 31, 2015 12:52 88 rows in about 4 hours
single run snapshot draft scrape transformer failed on September 04, 2015 01:34 88 rows in about 4 hours
single run snapshot draft scrape failed on September 04, 2015 08:08 64 rows in about 2 hours
single run snapshot draft scrape succeeded on September 04, 2015 23:26 88 rows in about 4 hours
single run snapshot draft scrape succeeded on February 04, 2016 10:35 95 rows in 3 minutes

Config

{
  "bot_id": "do-valores",
  "title": "Financial Licences of the Dominican Republic",
  "description": "This bot retrieves information about licensed securities firms in the Dominican Republic",
  "language": "python",
  "data_type": "primary data",
  "identifying_fields": [
    "Número Registro"
  ],
  "files": [
    "scraper.py",
    "licence.py"
  ],
  "frequency": "monthly",
  "publisher": {
    "name": "Superintendencia de Valores de la República Dominicana",
    "url": "http://siv.gov.do",
    "terms": "Copyright, all rights reserved",
    "terms_url": "http://siv.gov.do/terminos-de-uso/"
  },
  "transformers": [
    {
      "file": "licence.py",
      "data_type": "licence",
      "identifying_fields": [
        "licence_number"
      ]
    }
  ]
}