title Bot for South African Reserve Bank (mission_751_sarb) running
description This bot scrapes financial licences from a website
current run state scraping or awaiting scrape, docker status
last run single run snapshot 1 prescrape scrape succeeded on February 23, 2017 10:46
next run enqueued for a run, docker status
created by astrophe717 (Celyn Birkinshaw)
last reviewed by
State changed to running for run #10122, snapshot 2
commented about 7 years ago

A run started

State changed to scheduled for run #10122, snapshot 2
commented about 7 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
astrophe717 commented about 7 years ago

State changed to ingesting_data for run #10100, snapshot 1
commented about 7 years ago

The run's output is being ingested

State changed to storing_data for run #10100, snapshot 1
commented about 7 years ago

The run's output is being stored

State changed to running for run #10100, snapshot 1
Alex Skene commented about 7 years ago

The run has been restarted as a prescrape

State changed to storing_data for run #10099, snapshot 1
commented about 7 years ago

The run's output is being stored

State changed to running for run #10099, snapshot 1
Alex Skene commented about 7 years ago

The bot was accepted; starting run (using prescrape functionality) to ingest reviewed data

Bot update
commented about 7 years ago

A manual prescrape using data from /oc/openc/scrapers/output/draft/m/mission_751_sarb/10098 was started

State changed to under_final_review for run #10098, snapshot draft
commented about 7 years ago

A draft run succeeded; sending for final review

State changed to final_draft_running for run #10098, snapshot draft
Alex Skene commented about 7 years ago

A moderator has approved the draft bot; running a full draft for final review

Saved vars cleared
astrophe717 commented about 7 years ago

State changed to under_draft_review for run #10097, snapshot draft
Alex Skene commented about 7 years ago

A moderator has started reviewing the draft bot

State changed to pending_draft_review for run #10097, snapshot draft
commented about 7 years ago

Run succeeded; sending for draft review

State changed to draft_running for run #10097, snapshot draft
astrophe717 commented about 7 years ago

The bot was pushed; scheduling a draft run

State changed to pending_draft_review for run #10019, snapshot draft
commented about 7 years ago

Run succeeded; sending for draft review

State changed to draft_running for run #10019, snapshot draft
astrophe717 commented about 7 years ago

The bot was pushed; scheduling a draft run

State changed to pending_draft_review for run #9721, snapshot draft
commented over 7 years ago

Run succeeded; sending for draft review

State changed to draft_running for run #9721, snapshot draft
astrophe717 commented over 7 years ago

The bot was pushed; scheduling a draft run

State changed to draft_failed for run #9718, snapshot draft
commented over 7 years ago

A draft run failed

State changed to draft_running for run #9718, snapshot draft
astrophe717 commented over 7 years ago

The bot was pushed; scheduling a draft run

State changed to draft_failed for run #9679, snapshot draft
commented over 7 years ago

A draft run failed

State changed to draft_running for run #9679, snapshot draft
astrophe717 commented over 7 years ago

The bot was pushed; scheduling a draft run

Run history

event metadata
single run snapshot draft scrape failed on December 19, 2016 11:38 0 rows in less than a minute
single run snapshot draft scrape failed on December 23, 2016 10:39 0 rows in less than a minute
single run snapshot draft scrape succeeded on December 23, 2016 14:15 72 rows in 3 minutes
single run snapshot draft scrape succeeded on January 30, 2017 11:57 72 rows in 3 minutes
single run snapshot draft scrape succeeded on February 20, 2017 18:45 72 rows in 3 minutes
single run snapshot final draft scrape succeeded on February 21, 2017 11:01 72 rows in 3 minutes
single run snapshot 1 prescrape scrape succeeded on February 21, 2017 17:23 72 rows in less than a minute
single run snapshot 1 prescrape scrape succeeded on February 23, 2017 10:46 0 rows in less than a minute
single run snapshot 2 scrape scheduled on March 23, 2017 10:45 0 rows

Config

{
  "bot_id": "mission_751_SARB",
  "title": "Bot for South African Reserve Bank",
  "description": "This bot scrapes financial licences from a website",
  "language": "python",
  "data_type": "primary data",
  "identifying_fields": [
    "Institution",
    "CEO",
    "source_url"
  ],
  "files": [
    "scraper.py",
    "LICENSE.txt",
    "licence_transformer.py"
  ],
  "frequency": "monthly",
  "publisher": {
    "name": "South African Reserve Bank",
    "url": "https://www.resbank.co.za",
    "terms": "Operating",
    "terms_url": "https://www.resbank.co.za/AboutUs/CorporateCitizenship/Pages/CorporateCitizenship-Home.aspx"
  },
  "transformers": [
    {
      "file": "licence_transformer.py",
      "data_type": "licence",
      "identifying_fields": [
        "source_url",
        "licence_holder.entity_properties.name",
        "permissions.activity_name"
      ]
    }
  ]
}