title Insurance in Nepal (np-insurance) running
description This bot returns data relating to companies holding financial licences to carry out insurance activities in Nepal.
current run state scraping or awaiting scrape, docker status
last run single run snapshot 22 scrape succeeded on March 04, 2017 09:11
next run enqueued for a run, docker status
created by dinotash (Tom Curtis)
last reviewed by peter.evans
State changed to running for run #10125, snapshot 23
commented about 7 years ago

A run started

State changed to scheduled for run #10125, snapshot 23
commented about 7 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented about 7 years ago

State changed to ingesting_data for run #10053, snapshot 22
commented about 7 years ago

The run's output is being ingested

State changed to storing_data for run #10053, snapshot 22
commented about 7 years ago

The run's output is being stored

State changed to running for run #10053, snapshot 22
commented about 7 years ago

A run started

State changed to scheduled for run #10053, snapshot 22
commented about 7 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented about 7 years ago

State changed to ingesting_data for run #9820, snapshot 21
commented about 7 years ago

The run's output is being ingested

State changed to storing_data for run #9820, snapshot 21
commented about 7 years ago

The run's output is being stored

State changed to running for run #9820, snapshot 21
commented about 7 years ago

A run started

State changed to scheduled for run #9820, snapshot 21
commented over 7 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented over 7 years ago

State changed to ingesting_data for run #9539, snapshot 20
commented over 7 years ago

The run's output is being ingested

State changed to storing_data for run #9539, snapshot 20
commented over 7 years ago

The run's output is being stored

State changed to running for run #9539, snapshot 20
commented over 7 years ago

A run started

State changed to scheduled for run #9539, snapshot 20
commented over 7 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented over 7 years ago

State changed to ingesting_data for run #9299, snapshot 19
commented over 7 years ago

The run's output is being ingested

State changed to storing_data for run #9299, snapshot 19
commented over 7 years ago

The run's output is being stored

State changed to running for run #9299, snapshot 19
commented over 7 years ago

A run started

State changed to scheduled for run #9299, snapshot 19
commented over 7 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented over 7 years ago

State changed to ingesting_data for run #9028, snapshot 18
commented over 7 years ago

The run's output is being ingested

State changed to storing_data for run #9028, snapshot 18
commented over 7 years ago

The run's output is being stored

State changed to running for run #9028, snapshot 18
commented over 7 years ago

A run started

State changed to scheduled for run #9028, snapshot 18
commented over 7 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented over 7 years ago

State changed to ingesting_data for run #8748, snapshot 17
commented over 7 years ago

The run's output is being ingested

State changed to storing_data for run #8748, snapshot 17
commented over 7 years ago

The run's output is being stored

State changed to running for run #8748, snapshot 17
commented over 7 years ago

A run started

State changed to scheduled for run #8748, snapshot 17
commented over 7 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented over 7 years ago

State changed to ingesting_data for run #8436, snapshot 16
commented over 7 years ago

The run's output is being ingested

State changed to storing_data for run #8436, snapshot 16
commented over 7 years ago

The run's output is being stored

State changed to running for run #8436, snapshot 16
commented over 7 years ago

A run started

State changed to scheduled for run #8436, snapshot 16
commented over 7 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented over 7 years ago

State changed to ingesting_data for run #8055, snapshot 15
commented over 7 years ago

The run's output is being ingested

State changed to storing_data for run #8055, snapshot 15
commented over 7 years ago

The run's output is being stored

State changed to running for run #8055, snapshot 15
commented over 7 years ago

A run started

State changed to scheduled for run #8055, snapshot 15
commented almost 8 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented almost 8 years ago

State changed to ingesting_data for run #7648, snapshot 14
commented almost 8 years ago

The run's output is being ingested

State changed to storing_data for run #7648, snapshot 14
commented almost 8 years ago

The run's output is being stored

State changed to running for run #7648, snapshot 14
commented almost 8 years ago

A run started

State changed to scheduled for run #7648, snapshot 14
commented almost 8 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented almost 8 years ago

State changed to ingesting_data for run #7389, snapshot 13
commented almost 8 years ago

The run's output is being ingested

State changed to storing_data for run #7389, snapshot 13
commented almost 8 years ago

The run's output is being stored

State changed to running
commented almost 8 years ago

For run #7389:
A run started

State changed to scheduled
commented almost 8 years ago

For run #7389:
A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented almost 8 years ago

State changed to ingesting_data
commented almost 8 years ago

For run #6787:
The run's output is being ingested

State changed to storing_data
commented almost 8 years ago

For run #6787:
The run's output is being stored

State changed to running
commented almost 8 years ago

For run #6787:
A run started

Bot state update
commented about 8 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented about 8 years ago

Bot state update
commented about 8 years ago

A run finished; its output is now being processed

Bot state update
commented about 8 years ago

A run started

Bot state update
commented about 8 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented about 8 years ago

Bot state update
commented about 8 years ago

A run started

Bot state update
commented about 8 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented about 8 years ago

Bot state update
commented about 8 years ago

A run started

Bot state update
commented over 8 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented over 8 years ago

Bot state update
commented over 8 years ago

A run started

Bot state update
commented over 8 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented over 8 years ago

Bot state update
commented over 8 years ago

A run started

Bot state update
commented over 8 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented over 8 years ago

Bot state update
commented over 8 years ago

A run started

Bot state update
commented over 8 years ago

A snapshot completed; scheduling the first run of the next snapshot

Saved vars cleared
dinotash commented over 8 years ago

Bot state update
commented over 8 years ago

A run started

Bot state update
commented over 8 years ago

A snapshot completed; scheduling the first run of the next snapshot

Bot state update
commented over 8 years ago

A run started

Bot state update
commented over 8 years ago

A snapshot completed; scheduling the first run of the next snapshot

Bot state update
inglesp commented over 8 years ago

A failed run was restarted by the moderator

Bot state update
commented over 8 years ago

A run failed

Bot state update
commented over 8 years ago

A run started

Bot state update
commented almost 9 years ago

A run succeeded; scheduling the next run

Bot state update
commented almost 9 years ago

A run started

Bot state update
commented almost 9 years ago

A run succeeded; scheduling the next run

Bot state update
peter.evans commented almost 9 years ago

The bot was accepted; starting run to ingest reviewed data

Bot state update
commented almost 9 years ago

A draft run succeeded; sending for final review

Bot state update
peter.evans commented almost 9 years ago

A moderator has approved the draft bot; running a full draft for final review

Bot state update
peter.evans commented almost 9 years ago

A moderator has started reviewing the draft bot

(no subject)
peter.evans commented almost 9 years ago

Hi Tom,
Thanks for pushing the change, I'll have a look at it later today. The English/Nepali list threw me at first as well, I think Google Translate may have translated some of the company names literally & as you say it didn't seem to line up. Confidence is high, it's lowered when we are making inferences or parsing text such that errors might creep in.
Hope you're having a good weekend,
Peter

Re: (turbot bot [np-insurance])
dinotash commented almost 9 years ago

Hi Peter
I had another look and am fairly confident now that you were right - English on the left and Nepali on the right. I’ve combined these into one record per company.
I’ve also added confidence = HIGH to the licence transformer and pushed an updated bot.
Tom

Bot state update
commented almost 9 years ago

Run succeeded; sending for draft review

Bot state update
dinotash commented almost 9 years ago

The bot was pushed; scheduling a draft run

Re: (turbot bot [np-insurance])
dinotash commented almost 9 years ago

Hi Peter
It was a deliberate choice not to combine the English and Nepalese names because when I first saw the page, it looked like they were separate lists. When I google translated some of the Nepalese names they didn’t line up with the English ones, and there seemed to be more in that list. However, looking again, it may just be because the Nepalese text is more spread out vertically. I’ll check more carefully and update the bot if needed.
What would you suggest for the confidence field? High as it’s a fairly simple site to scrape?
Thanks
Tom

(no subject)
peter.evans commented almost 9 years ago

Hi Tom,
Thank you for writing this bot scraping Nepalese insurance licences. I’m reviewing the output now - Generally the bot looks to be in good shape, there are a couple of things that we could tweak to get it outputting perfectly:
For this page: http://www.bsib.org.np/index.php?option=cms&id=38 I believe the English company names are listed on the left and Nepali translations are listed on the right. Do you think that is right? At the moment we’re scraping each of these into different records so I think we should probably combine them, we can include the translation in the scraper & use the English company name for the transformer.
Ah I see that’s what you’ve done for the other pages actually so should be a simple fix, or was there a reason for addressing this page differently?
We could also add a confidence field to the licence transformer.
Otherwise that’s it, the bot looks great. Once we’ve looked into those issues it should be simple to get it accepted and ingested into OpenCorporates.
Thanks,
Peter

Bot state update
commented about 9 years ago

A draft run succeeded; sending for review

Bot state update
commented about 9 years ago

A draft run started

Bot state update
dinotash commented about 9 years ago

The bot was pushed; scheduling a draft run

Run history

event metadata
single run snapshot draft scrape succeeded on March 03, 2015 20:21 144 rows in 1 minute
single run snapshot draft scrape succeeded on May 24, 2015 09:28 64 rows in less than a minute
single run snapshot final draft scrape succeeded on May 24, 2015 18:25 64 rows in less than a minute
single run snapshot 1 prescrape scrape succeeded on May 24, 2015 18:26 64 rows in less than a minute
single run snapshot 2 scrape succeeded on June 24, 2015 18:27 64 rows in less than a minute
single run snapshot 3 scrape failed on July 24, 2015 18:26 0 rows in less than a minute
single run snapshot 3 scrape succeeded on August 04, 2015 09:11 64 rows in less than a minute
single run snapshot 4 scrape succeeded on September 04, 2015 09:11 64 rows in less than a minute
single run snapshot 5 scrape succeeded on October 04, 2015 09:13 64 rows in 2 minutes
single run snapshot 6 scrape succeeded on November 04, 2015 09:14 64 rows in 3 minutes
single run snapshot 7 scrape succeeded on December 04, 2015 09:11 64 rows in less than a minute
single run snapshot 8 scrape succeeded on January 04, 2016 09:12 64 rows in less than a minute
single run snapshot 9 scrape succeeded on February 04, 2016 09:12 64 rows in 1 minute
single run snapshot 10 scrape succeeded on March 04, 2016 09:11 64 rows in less than a minute
single run snapshot 11 scrape succeeded on April 04, 2016 09:12 64 rows in 1 minute
single run snapshot 12 scrape succeeded on May 04, 2016 09:11 64 rows in less than a minute
single run snapshot 13 scrape succeeded on June 04, 2016 09:12 64 rows in 1 minute
single run snapshot 14 scrape succeeded on July 04, 2016 09:11 64 rows in less than a minute
single run snapshot 15 scrape succeeded on August 04, 2016 09:11 64 rows in less than a minute
single run snapshot 16 scrape succeeded on September 04, 2016 09:11 64 rows in less than a minute
single run snapshot 17 scrape succeeded on October 04, 2016 09:11 64 rows in less than a minute
single run snapshot 18 scrape succeeded on November 04, 2016 09:11 64 rows in less than a minute
single run snapshot 19 scrape succeeded on December 04, 2016 09:11 64 rows in less than a minute
single run snapshot 20 scrape succeeded on January 04, 2017 09:11 64 rows in less than a minute
single run snapshot 21 scrape succeeded on February 04, 2017 09:11 65 rows in less than a minute
single run snapshot 22 scrape succeeded on March 04, 2017 09:11 65 rows in less than a minute
single run snapshot 23 scrape scheduled on April 04, 2017 09:11 0 rows

Config

{
  "bot_id": "np-insurance",
  "title": "Insurance in Nepal",
  "description": "This bot returns data relating to companies holding financial licences to carry out insurance activities in Nepal.",
  "language": "python",
  "data_type": "primary data",
  "identifying_fields": [
    "name"
  ],
  "files": [
    "scraper.py",
    "licence.py"
  ],
  "company_fields": {
    "name": "name"
  },
  "frequency": "monthly",
  "publisher": {
    "name": "Insurance Regulatory Authority of Nepal",
    "url": "http://www.bsib.org.np",
    "terms": "Not stated",
    "terms_url": "http://www.bsib.org.np/"
  },
  "transformers": [
    {
      "file": "licence.py",
      "data_type": "simple-licence",
      "identifying_fields": [
        "company_name"
      ]
    }
  ]
}