harry/News-ML-Decoder
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Crawl log: precise crawl finished (523 more requests, 46,538 headlines)

Browse Source
This commit is contained in:
Harwinder Sekhon
2026-09-27 19:46:13 -04:00
parent 9273e41d56
commit 29ee798b2a
1 changed files with 4 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
logs/crawl_precise.log
+4
View File
@@ -39,3 +39,7 @@ precise: 200 windows of 14 days x 21 queries; 729 done, 4039 to go
precise: 200 windows of 14 days x 21 queries; 6125 done, 523 to go precise: 200 windows of 14 days x 21 queries; 6125 done, 523 to go
100 requests, window 2024-03-25, 44823 unique headlines, 27 req/min 100 requests, window 2024-03-25, 44823 unique headlines, 27 req/min
200 requests, window 2021-10-11, 45308 unique headlines, 27 req/min 200 requests, window 2021-10-11, 45308 unique headlines, 27 req/min
300 requests, window 2022-07-04, 45544 unique headlines, 27 req/min
400 requests, window 2024-01-01, 45842 unique headlines, 27 req/min
500 requests, window 2026-06-15, 46461 unique headlines, 27 req/min
done: 523 requests this run, 46538 unique headlines in /home/harwinder/Development/ML-NEWS/data/history.db