Data Scraping Specialist

Welcome to BGaming! We're a fun-loving, globally-recognized game development studio, best known for creating captivating video slots that players just can't get enough of. Our games are a delightful blend of stunning visuals, engaging features, and thrilling storylines, all wrapped up in a user-friendly package that works seamlessly on any device.

But we're not just about good looks – we're pioneers too! We were the first major iGaming provider to introduce the "provably fairness feature", using cutting-edge cryptography to ensure that our players can trust the results of their games to be truly random and unbiased.

Role Overview:

This is a temporary, project-based assignment. The engagement will last until the project is completed, which means achieving full coverage of all required parsers.

Key points:

• The list of target resources will be provided and may change during the project, depending on priorities and business needs.
• The role is focused on delivering stable, automated data collection tools for each assigned resource.
• Work is considered complete once all required parsers are implemented, tested, and integrated into the existing framework.
• Collaboration will be milestone-oriented, with clear deliverables for each stage.

Responsibilities:

• Build and maintain resilient data scrapers
• Bypass anti-bot controls (WAF, CAPTCHA) using smart proxy rotation and a stable browser fingerprint
• Handle dynamic content: infinite scroll, “Load more,” lazy images, filters/menus
• Ensure data quality (≥95% of key fields filled), idempotency, and de-duplication by (url, title).
• Log, measure, and alert: counters for blocks, CAPTCHAs, retries, proxy switches; produce daily summaries
• Deliver datasets (CSV/JSON/Parquet) with a short summary; package solutions in Docker; manage config/secrets via .env.
• Document adapters and onboard new target sites quickly.

Requirements:

• Python 3.10+; libraries: Playwright (preferred) and/or Selenium (+ undetected-chromedriver), BeautifulSoup/lxml, httpx/requests, pandas.
• Anti-bot / WAF bypass: understanding of rate limiting, cookies/canary tokens, JS challenges (Cloudflare/Imperva), proper headers/HTTP2, and fingerprint control (User-Agent, languages, timezone, WebGL/Canvas, fonts).
  
• Smart rotating proxies: residential/mobile pools, rotation by country/ASN, sticky sessions, proxy health checks; experience with providers like Bright Data/Oxylabs and cloud anti-bot gateways (ZenRows/ScrapeNinja or similar).
  
• CAPTCHA bypass: integrations with 2Captcha/Anti-Captcha/CapMonster; handling reCAPTCHA/hCaptcha; timeouts, retries, and fallback logic.
  
• Asynchronous execution: asyncio, queues/pipelines, backpressure; retries with exponential backoff + jitter.
  
• Dynamic DOM/XHR handling: robust waits and selectors (data-attributes), scroll/click scenarios; extract data from XHR/GraphQL/embedded JSON.
  
• Data layer: currency/date normalization, validation (Pydantic preferred), idempotent loads; export to CSV/JSON/Parquet.
  
• Observability: structured logs (INFO/WARN/ERROR) and metrics (success/block/captcha/proxy_switch/latency).
  
• Infrastructure: Docker (+ docker-compose), Git/CI, secret management via .env/vault; code review and documentation (README, flow diagrams).
  
• Compliance: respect site ToS/robots.txt and legality of collection; adhere to privacy rules (GDPR/cookie policies).

Nice to have:

• Experience with Scrapy or other structured scraping frameworks (beyond Playwright/Selenium).
• Familiarity with Scrapoxy or similar proxy orchestration tools.
• Exposure to distributed job queues (Celery, Airflow, Kafka).
• Knowledge of advanced fingerprinting evasion (puppeteer-stealth, browser patching).
• Hands-on with multi-locale data collection (currency, timezone, i18n parsing).
• Background in data quality pipelines (deduplication, anomaly detection).
• Experience deploying at scale in Kubernetes or serverless environments.
• Familiarity with Prometheus / Grafana / ELK for monitoring.
• Participation in open-source scraping or security projects.

Work Conditions:

• Full-time flexible schedule (remote work is possible);
• Highly competitive compensation, motivation, benefits, and social programs;
• Paid vacation + 4 sick + 1 day off;
• Medical Insurance;
• Professional training & certifications, language courses;
• Enjoy working in a fast-growing modern iGaming company;
• Great community of experts and industry professionals.

‍