Articles by Python | datawookie

Selenium Template #1: Docker Host Network

April 17, 2021 | Python | datawookie

This post will show you how to set up the following: a Selenium instance and a simple script connecting to Selenium. Both of these will run in Docker containers and will communicate over the host network. This is part of a series of posts: Part 1: Selenium Template — Docker Host Network ...
[...Read more...]

Selenium Crawler #1: Docker Host Network

April 17, 2021 | Python | datawookie

This post will show you how to set up the following: a Selenium instance and a simple script connecting to Selenium. Both of these will run in Docker containers and will communicate over the host network. Selenium Service Create a Selenium container, exposing port 4444 on the host. This means that ...
[...Read more...]

Launching Selenium with JavaScript Disabled

February 3, 2021 | Python | datawookie

I have a rather obscure situation where I want to launch Selenium… but with JavaScript disabled. from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.common.desired_capabilities import DesiredCapabilities After a bit of Googling it turns out that this is eminently possible by just ...
[...Read more...]

Persistent Selenium Sessions

January 28, 2021 | Python | datawookie

I have a project where I need to have a persistent Selenium session. There’s a script which will leave a browser window open when it exits. When the script runs again it should connect to the same window. Derive a new class from selenium.webdriver.Remote. from selenium import ...
[...Read more...]

Retail Data: Scraping & API

March 15, 2020 | Python | datawookie

I’ve been wanting to gather data on retail prices for quite some time. Finally, just before Christmas 2019, I had some time on my hands, so I started to put something together. The Plan This was the plan: a fleet of scrapers, each focusing on a specific retailer; the scrapers ...
[...Read more...]

Sliding Puzzle Solvable?

April 10, 2019 | Python | datawookie

I’m helping develop a new game concept, which is based on the sliding puzzle game. The idea is to randomise the initial configuration of the puzzle. However, I quickly discovered that half of the resulting configurations were not solvable. Not good...
[...Read more...]

Python: First Steps with MongoDB

September 28, 2016 | Python | datawookie

I’m busy working my way through Kyle Banker’s MongoDB in Action. Much of the example code in the book is given in Ruby. Despite the fact that I’d love to learn more about Ruby, for the moment it makes more sense for me to follow along with ...
[...Read more...]
1 2 3