WIP: scrapy + multiprocessing#7

Open

lhaarman wants to merge 3 commits into

mainfrom

experimental/scrapy_multiprocessing

Commits on May 20, 2026

Add configuration for skippable domains such as social media
lhaarman
committed

Commits on May 21, 2026

Basic setup scrapy with multiprocessing; TODO testing
lhaarman
committed

Commits on May 28, 2026

temp commit; saving result
lhaarman
committed