Requestium is a python library that merges the power of Requests, Selenium, and Parsel into a single integrated tool for automatizing web actions.
Requestium adds independent improvements to both Requests and Selenium, and every new feature is lazily evaluated, so its useful even if writing scripts that use only Requests or Selenium.
- Enables switching between a Requests’ Session and a Selenium webdriver while maintaining the current web session.
- Integrates Parsel’s parser into the library, making xpath, css, and regex much cleaner to write.
- Improves Selenium’s handling of dynamically loading elements.
- Makes cookie handling more flexible in Selenium.
- Makes clicking elements in Selenium more reliable.
- Supports Chrome and PhantomJS.
All code examples below are labeled for reference. They are purely intended to provide examples of concepts. Most of them can be written in a much better way.
Headless Chrome is a way to run the Chrome browser in a headless environment. Essentially, running Chrome without chrome! It brings all modern web platform features provided by Chromium and the Blink rendering engine to the command line.
Why is that useful?
A headless browser is a great tool for automated testing and server environments where you don’t need a visible UI shell. For example, you may want to run some tests against a real web page, create a PDF of it, or just inspect how the browser renders an URL.
Without your consent most major web platforms leak whether you are logged in. This allows any website to detect on which platforms you’re signed up. Since there are lots of platforms with specific demographics an attacker could reason about your personality, too.
For the sake of simplicity, let’s assume your everyday framework is React but if you use something else, replace React with X framework because this will still apply to you too.
“This tool turns a curl command into Go code. (To do the reverse, check out sethgrid/gencurl.) Currently, it knows the following options: -d/–data, -H/–header, -I/–head, -u/–user, and -X/–request. It also understands JSON content types. As always, be sure to go fmt your code and pay attention to imports. There’s probably bugs; please contribute on GitHub! For a quick way to generate a Go struct from JSON, see JSON-to-Go…”