title cssf.lu bot (supervisedentities-cssf-lu) draft_failed
description CSSF Luxembourg scraper.
current run state not running
last run single run snapshot draft scrape failed on February 03, 2015 19:20
next run n/a
created by julochrobak (Julius Chrobak)
last reviewed by peter.evans
(no subject)
peter.evans commented over 5 years ago

Hi Julius,
Thanks for coming to the Zurich #FlashHacks, it sounded great! I'm having a look at the bots that were pushed now in order to get them accepted and onto the system.
Your CCF Luxembourg scraper looks good, I've noticed that some of the numerical data is being output as numbers rather than valid JSON, could we edit that to output strings instead? Before that though could you please try doing a "turbot bots:validate" and let me know if you receive a sensible error message as we're hoping to catch that sort of thing at validation.
Thank you,
Peter

Bot state update
commented over 5 years ago

A run failed

Bot state update
commented over 5 years ago

A draft run started

Bot state update
julochrobak commented over 5 years ago

The bot was pushed; scheduling a draft run

Run history

event metadata
single run snapshot draft scrape failed on February 03, 2015 19:20 2002 rows in 13 minutes

Config

{
  "bot_id": "supervisedentities-cssf-lu",
  "title": "cssf.lu bot",
  "description": "CSSF Luxembourg scraper.",
  "language": "python",
  "data_type": "primary data",
  "identifying_fields": [
    "licence_number"
  ],
  "files": [
    "scraper.py",
    "licence_transformer.py"
  ],
  "frequency": "monthly",
  "transformers": [
    {
      "file": "licence_transformer.py",
      "data_type": "simple-licence",
      "identifying_fields": [
        "licence_number"
      ]
    }
  ],
  "publisher": {
    "name": "Commision de Surveillance de Secteur Financier",
    "url": "http://supervisedentities.cssf.lu",
    "terms": "All rights reserved. Copyright by Commission de Surveillance du Secteur Financier, Luxembourg"
  }
}