Puppeteer: Quick Start

This post is a snippet of the official Puppeteer guide, plus my personal opinion.

https://developers.google.com/web/tools/puppeteer/get-started


What is Puppeteer?

"Puppeteer is a Node library which provides a high-level API to control headless Chrome or Chromium over the DevTools Protocol. It can also be configured to use full (non-headless) Chrome or Chromium" (https://developers.google.com/web/tools/puppeteer).


Selenium vs Puppeteer

A friend of mine who runs a crowler-based service told me he usually uses

  • Selenium in Java for production and
  • Puppeteer for a small project with few people.

You can also check Selenide if you want to compare the technologies: https://web-quickstart.blogspot.com/2021/03/selenide.html


Install yarn (if not installed yet)

You can also use npm instead. For me, yarn looks easier if you run it on Mac.

% brew install yarn
% yarn -v
1.22.10


Install Puppeteer

% yarn add puppeteer
% yarn add puppeteer-core


Run

Create example.js as:

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://example.com');
  await page.screenshot({ path: 'example.png' });

  await browser.close();
})();

% node example.js
This will save a screenshot of https://example.com as example.png:


Comments

Post a Comment

Popular posts from this blog

Selenide: Quick Start

Selenide is a Selenium wrapper in Java for testing.   This post describes how you can start using Selenide (using Gradle). 1. Setup Java Application # keep pressing enters to choose default values gradle init --type java-application # confirm that test succeeds gradle test 2. Add Selenide Dependency open build.gradle and edit as: dependencies {   ...   testImplementation 'com.codeborne:selenide:5.13.0' } 3. Write Test (replace the existing test with selenide code) src/test/java/selenide/AppTest.java package selenide; import static com.codeborne.selenide.Condition.text; import static com.codeborne.selenide.Selenide.$; import static com.codeborne.selenide.Selenide.open; import org.junit.Test;   public class AppTest { @Test public void test() { open("https://www.google.ca/"); $("input[type=text]").val("test").pressEnter(); $("body").shouldHave(text("test")); } } 4. Run Selenide gradle test This will open a browser t...
Read more

Minikube Installation for M1 Mac

Image
In this post, I am going to try Minikube on Mac with M1 Chip. what is minikube Minikube is k8s tool running on a local machine. See also kind  for multi cluster implementation for local k8s. install % curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-darwin-arm64 % install minikube-darwin-arm64 /usr/local/bin/minikube % minikube version  minikube version: v1.20.0  Note: Installing with brew fails due to a different binary as of 2021-05-24. % brew install minikube % minikube version ❌  Exiting due to MK_WRONG_BINARY_M1: You are trying to run amd64 binary on M1 system. Please use darwin/arm64 binary instead (Download at https://github.com/kubernetes/minikube/releases/download/v1.20.0/minikube-darwin-amd64.) % brew uninstall minikube start The default driver 'virtualbox' is not supported on M1, but you can use 'docker' driver:  https://minikube.sigs.k8s.io/docs/drivers/docker/  . Make sure docker is running, and then specify the driver ...
Read more

Testinfra: Quick Start - Run Sample Test to EC2 server

In this post, I am going to try Testinfra . what is Testinfra? Testinfra is a server-testing framework in Python. https://testinfra.readthedocs.io/en/latest/ Similar technologies include: Serverspec, InSpec, and Goss. test server I have create a test server on EC2 as follows: ssh -i test ec2-user@100.26.216.164 https://web-quickstart.blogspot.com/2021/04/terraform-ec2-instance-with-ssh-setup.html install % python --version Python 3.8.5 % pip --version pip 20.2.4 % pip install pytest-testinfra % pip list | grep infra        pytest-testinfra                   6.3.0 run locally (test Mac) % cat root.py def test_passwd_file(host):     passwd = host.file("/etc/passwd")     assert passwd.contains("root")     assert passwd.user == "root" % py.test root.py run remotely (test EC2) Prepare ssh config like below: % cat config  Host testserver   HostName 100.26.216.164   Port 22  ...
Read more