Some websites require that you execute javascript before you can see any content on the page, or you may want to emulate a real user. In order to handle these use cases we'll learn how to use pyppeteer which is a library for controlling a Headless Chrome ... (more…)

Exploring Python through counter-intuitive snippets. If you think you know Python, think once more! - satwikkansal/wtfpython... (more…)

Last year I started working on a 'Big Data' exercise. It's an ongoing project that mixes large amounts of web traffic, data ingestion and analytics. It's also really fun. We get to play with an array of new technologies - sometimes on a bet, granted - but...

The scraping series will not get completed without discussing Scrapy. In this post I am going to write a web crawler that will scrape data from OLX's Electronics & Appliances' items. Before I get into the code, how about having a brief intro of Scrap...

When I wrote about getting the Python LSP server working with venvs in a brute force way, Ian Z aka nobrowser commented (and I'm going to quote rather than paraphrase): (more…)