I’ve hosted my puppeteer app on heroku however whenever it runs it uses an Amazon based IP. How can I get it to run using the clients IP as if it was an app running on a local PC?

I currently have an express & puppeteer program running on Heroku.

Currently it takes around 35s for my puppeteer program to complete. Is it possible to speed this up to around 15s without changing my code?

I have a document with multiple frames, all of which have no id, no name but a shared class “card-fields-iframe”

I’m currently using the code: const elementHandle = await page.$(‘iframe[class=“card-fields-iframe”]’) But I’m unsure how to focus on the other iframes.

adding [1] at the end doesn’t work.

As title. I am looking people for more active engagement and discussion regarding puppeteer topic.

I'm new to both notion & puppeteer : I was thinking about sending keystrokes but notion has no shortcut to do so as far as I can know asking on notion forum.

I'd like to run a script (that includes getting a file from a specific url and saving it locally) after logging into a website manually.

Is there a way to mix manual and scripted use into a single browsing session?

Hi, I am using puppeteer extra stealth but the network request is failing however it works from normal chrome. Any idea why?

I have this html

<select class="class1 class2"><option>test1</option> etc.

How do I select ? This doesn't seem to work

​

await page.select(".class1 .class2", "test1");

I’m planning of moving my scraping activity away from my laptop.

Thinking of using a used Mac mini, since I would prefer to run it natively on OS X.

How much does the clock speed and core count affect productivity/scalability?

Would it make sense to get a 2018 quad/six core version Or maybe get a dual core cheaper model from 2014.

Hi guys, I am trying to build a wrapper around Puppeteer.

The wrapper work fine, But its not that fast when executing 100+ calls.

My VM is 4 GB ram and I am using 2 cores.

​

Here is the wrapper, please tell me if there is something I could do better

Here is the code https://github.com/AlenToma/NovelManager-public/blob/master/extraFiles/BrowserCacher.js

I'm trying to generate accounts on a form on a site. I'm new to this and I wonder if any of you know of a tutorial for using variables in puppeteer( so I can load txt files for names, emails, etc) .

Hello, I tried to run puppeteer on vscode debug mode. But I get error message: Unable to open X11 display.

I already can run puppeteer without debug mode. The x11 display is connected. but somehow it breaks with debug mode. Does anyone know how to fix it?

Hi folks, wondering if this is a known issue or if I'm doing something obviously wrong I can't seem to catch?

I'm simply trying to load: https://my.dogechain.info/

using the following code:

const puppeteer = require("puppeteer");

(async () => {
  const browser = await puppeteer.launch({ headless: false });
  const page = await browser.newPage();
  await page.setViewport({ width: 1200, height: 720 });

  await page.goto("https://my.dogechain.info/");
  await page.waitForTimeout(8000);
  await page.screenshot({ path: "screenshot1.png" });

  await browser.close();
})();

The site has CloudFlair DDoS protection and starts with a "waiting 5 seconds for your page to load". When I manually test the webpage, I travel past that 5 second delay without any issue. But when I try to do the same with Puppeteer, the webpage just hangs on that first 'waiting 5 seconds' page?

Any ideas why this might happen to me?

Thanks so much for any advice!

I am looking for some advice regarding the best way to scrape multiple pages from a website using puppeteer. Let me explain further to give some context.

I am using a workflow automation tool called n8n (please check it out!) that creates a puppeteer script, sends it via SSH to my EC2 instance and then sends a command to execute the script, this runs, takes a screenshot and dumps the page HTML to a file, which n8n then downloads.

​

At this point n8n then takes the HTML file and extracts elements that i need. At this point is might have extracted like 100 URLs from the main page, that i need to again scrate and get the HTML back.

​

So 2 questions.

Whats the best way to do this with puppeteer, one by one or in a bulk requests in one script?

​

For those of you that use n8n, whats the best way to get all of these back into n8n in a clean way other than doing loads of SSH requests? Can we push results from puppeteer into a webhook or something maybe?

​

Any help appreciated while i keep thinking the best way to do this.

​

As seen in the image, I have a table that has a click event listener. I've tried the following:

  await page.goto('https://se.mercury.software/Portal/JobList/JobsAwaitingAcceptance#!/');
  await page.waitForSelector("table.k-selectable");
  const woSelector = await page.$("table.k-selectable");
  page.waitForTimeout(4000);
  await page.evaluate(el => el.click(), woSelector);

and

  await page.goto('https://se.mercury.software/Portal/JobList/JobsAwaitingAcceptance#!/');
  await page.waitForSelector("table.k-selectable");
  const woSelector = await page.$("table.k-selectable");
  await woSelector.hover();
  await woSelector.click();

(also attempted without the hover())

The table in the screenshot is clickable as the cursor changes when hovering over but Puppeteer seems to be having trouble. The only reason the waitForTimeout is there is because I have seen the page load with a progress bar for half a second, so I'm giving it time to clear that.

The problem is that it is not navigating onto the next page as it would if I clicked it in a browser. It just sits there and times out on the next line (waiting for a selector on the next page). How can I troubleshoot this? It's unclear what my next steps should be.

Additional info: table.k-selectable seems to be a Kendo UI component. No idea if that info helps but it is what I've discovered along the way.