BETA

Summary for scraper logs from the 2020-01-24

Summary of log messages

LevelCount
Exceptions2
error30
warning0
info22679
debug70412

Summary of changes

TypeAddedUpdated
mep 0 12
dossier 0 56
amendment 5364 0
mep_activity 741 0
comagenda 26 4
TimeModuleLevelMessage
2020-01-24T00:17:42.095401dossiererrorthis url returns a hard 404: http://www.europarl.europa.eu/oeil/popups/ficheprocedure.do?reference=2018/2633(RSP)&l=en
2020-01-24T00:17:59.340706dossiererrorthis url returns a hard 404: https://oeil.secure.europarl.europa.eu/oeil//popups/ficheprocedure.do?reference=2018/0063(COD)&l=en
2020-01-24T00:18:18.113145dossiererrorthis url returns a hard 404: http://www.europarl.europa.eu/oeil/popups/ficheprocedure.do?reference=2017/2840(RSP)&l=en
2020-01-24T00:18:21.597335dossiererrorthis url returns a hard 404: http://www.europarl.europa.eu/oeil/popups/ficheprocedure.do?reference=2012/2047(INI)&l=en
2020-01-24T00:19:28.846426dossiererrorthis url returns a hard 404: http://www.europarl.europa.eu/oeil/popups/ficheprocedure.do?reference=2015/2148(INI)&l=en
2020-01-24T00:20:39.019636dossiererrorthis url returns a hard 404: http://www.europarl.europa.eu/oeil/popups/ficheprocedure.do?reference=2017/2848(RSP)&l=en
2020-01-24T00:20:44.984279dossiererrorthis url returns a hard 404: http://www.europarl.europa.eu/oeil/popups/ficheprocedure.do?reference=2018/2669(RSP)&l=en
2020-01-24T00:20:52.660865dossiererrorno content found for section "Attention!" of 2015/2542(DEA)
2020-01-24T00:20:52.714913dossiererrorthis url returns a hard 404: http://www.europarl.europa.eu/oeil/popups/ficheprocedure.do?reference=2014/2230(INI)&l=en
2020-01-24T00:20:52.732306dossiererrorno content found for section "Delegated act was actually rejected!" of 2015/2542(DEA)
2020-01-24T00:21:42.047106dossiererrorthis url returns a hard 404: http://www.europarl.europa.eu/oeil/popups/ficheprocedure.do?reference=2016/0282(COD)&l=en
2020-01-24T00:21:42.174613dossiererrorthis url returns a hard 404: http://www.europarl.europa.eu/oeil/popups/ficheprocedure.do?reference=2015/2222(INI)&l=en
2020-01-24T00:21:43.397207dossiererrorthis url returns a hard 404: http://www.europarl.europa.eu/oeil/popups/ficheprocedure.do?reference=2017/2024(INL)&l=en
2020-01-24T00:23:01.240813dossiererrorthis url returns a hard 404: http://www.europarl.europa.eu/oeil/popups/ficheprocedure.do?reference=2016/0400(COD)&l=en
2020-01-24T00:26:00.643864dossiererrorthis url returns a hard 404: https://oeil.secure.europarl.europa.eu/oeil/popups/ficheprocedure.do?reference=2019/2110(INI)&l=en
2020-01-24T00:27:22.542009amendmenterrorcould not find non-empty line above pagebreak in http://www.europarl.europa.eu/doceo/document/LIBE-AM-646874_EN.pdf
2020-01-24T00:27:22.542555mgrerrorfailed to execute amendment job {'onfinished': {'daisy': True}, 'url': 'http://www.europarl.europa.eu/doceo/document/LIBE-AM-646874_EN.pdf', 'meps': 'Bettina VOLLATH'} (ValueError('no EN marker found: http://www.europarl.europa.eu/doceo/document/LIBE-AM-646874_EN.pdf'))
2020-01-24T00:27:22.606465mgrerrorTraceback (most recent call last):
File "/home/pt/pt2/scraper_service.py", line 63, in consume
    ret = scraper.scrape(**job)
  File "scrapers/amendment.py", line 653, in scrape
    text, PE=getraw(url)
  File "scrapers/amendment.py", line 55, in getraw
    return unpaginate(text,pdf)
  File "scrapers/amendment.py", line 269, in unpaginate
    raise ValueError("no EN marker found: %s" % url)
ValueError: no EN marker found: http://www.europarl.europa.eu/doceo/document/LIBE-AM-646874_EN.pdf
2020-01-24T00:27:23.764418amendmenterrorcould not find non-empty line above pagebreak in http://www.europarl.europa.eu/doceo/document/LIBE-AM-646877_EN.pdf
2020-01-24T00:27:23.785917mgrerrorfailed to execute amendment job {'onfinished': {'daisy': True}, 'url': 'http://www.europarl.europa.eu/doceo/document/LIBE-AM-646877_EN.pdf', 'meps': 'Bettina VOLLATH'} (ValueError('no EN marker found: http://www.europarl.europa.eu/doceo/document/LIBE-AM-646877_EN.pdf'))
2020-01-24T00:27:23.966036mgrerrorTraceback (most recent call last):
File "/home/pt/pt2/scraper_service.py", line 63, in consume
    ret = scraper.scrape(**job)
  File "scrapers/amendment.py", line 653, in scrape
    text, PE=getraw(url)
  File "scrapers/amendment.py", line 55, in getraw
    return unpaginate(text,pdf)
  File "scrapers/amendment.py", line 269, in unpaginate
    raise ValueError("no EN marker found: %s" % url)
ValueError: no EN marker found: http://www.europarl.europa.eu/doceo/document/LIBE-AM-646877_EN.pdf
2020-01-24T00:28:23.879390amendmenterror[!] couldn't find ref: Committee on Development
2020-01-24T00:28:24.982089amendmenterror[!] couldn't find ref: Committee on Development
2020-01-24T00:28:24.991248amendmenterrorcouldn't find dossier reference in source pdf: http://www.europarl.europa.eu/doceo/document/DEVE-AM-641421_EN.pdf
2020-01-24T00:28:26.960940amendmenterror[!] couldn't find ref: Committee on Development
2020-01-24T00:28:26.981448amendmenterrorcouldn't find dossier reference in source pdf: http://www.europarl.europa.eu/doceo/document/DEVE-AM-641422_EN.pdf
2020-01-24T00:29:36.210060amendmenterror[!] couldn't find ref: Committee on Economic and Monetary Affairs
2020-01-24T00:29:42.192528amendmenterror[!] couldn't find ref: Committee on Economic and Monetary Affairs
2020-01-24T00:30:21.047167amendmenterror[!] couldn't find ref: Committee on the Environment, Public Health and Food Safety
2020-01-24T00:32:42.808554amendmenterror[!] couldn't find ref: Committee on Development