0

環境

  • pyppeteer 1.0.2
  • Python 3.10.2
  • Ubuntu 22.04
    • VirtualBox v7.0.6 r155176を利用している
    • ホストOSはWindows11 Pro

問題点

pyppeteerでブラウザを操作したいです。
以下のコードを実行したところ、page.gotoを実行する前で、処理が止まりました。

質問

下記のコードが途中で処理が止まってしまう原因と解決策を教えていただきたいです。

以下に記載のログで、最終行のRECV: {"id":4,"result":{}}以降は、1分以上待ってもログは出力されませんでした。

補足

先日Ubuntu18.04から22.04にバージョンアップしました。
Ubuntu18.04のときは上記のようなコード(厳密に一緒ではないが)は動きました。
Ubuntuのバージョンアップにより、何かが変わったのでしょうか?


現状のコード:

import os
import asyncio
from pyppeteer import launch
from pyppeteer.browser import Browser
from pyppeteer.page import Page
from typing import Optional
import time

import logging

logger = logging.getLogger(__name__)

logging.basicConfig(format='%(levelname)-8s : %(asctime)s : %(name)s : %(message)s', level=logging.DEBUG)


async def main():
    logger.info("start 'launch()'")
    browser: Browser = await launch()
    logger.info("start 'newPage()'")
    page: Page = await browser.newPage()
    logger.info("start 'goto()'")
    page.goto("https://www.google.com/")
    logger.info(f"title='{page.title}'")
    await browser.close()


if __name__ == "__main__":
    asyncio.new_event_loop().run_until_complete(main())

ログ:

$ python sample.py
DEBUG    : 2023-02-10 02:40:36,722 : asyncio : Using selector: EpollSelector
INFO     : 2023-02-10 02:40:36,723 : __main__ : start 'launch()'
INFO     : 2023-02-10 02:40:36,962 : pyppeteer.launcher : Browser listening on: ws://127.0.0.1:44795/devtools/browser/41f9db60-def6-4e84-86bb-47a8238e1269
DEBUG    : 2023-02-10 02:40:36,962 : pyppeteer.connection.Connection : SEND: {"id": 1, "method": "Target.setDiscoverTargets", "params": {"discover": true}}
DEBUG    : 2023-02-10 02:40:36,963 : websockets.client : = connection is CONNECTING
DEBUG    : 2023-02-10 02:40:36,963 : websockets.client : > GET /devtools/browser/41f9db60-def6-4e84-86bb-47a8238e1269 HTTP/1.1
DEBUG    : 2023-02-10 02:40:36,963 : websockets.client : > Host: 127.0.0.1:44795
DEBUG    : 2023-02-10 02:40:36,964 : websockets.client : > Upgrade: websocket
DEBUG    : 2023-02-10 02:40:36,964 : websockets.client : > Connection: Upgrade
DEBUG    : 2023-02-10 02:40:36,964 : websockets.client : > Sec-WebSocket-Key: x249CRPQgAIEzSuRpwfwWA==
DEBUG    : 2023-02-10 02:40:36,964 : websockets.client : > Sec-WebSocket-Version: 13
DEBUG    : 2023-02-10 02:40:36,964 : websockets.client : > Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
DEBUG    : 2023-02-10 02:40:36,964 : websockets.client : > User-Agent: Python/3.10 websockets/10.4
DEBUG    : 2023-02-10 02:40:36,968 : websockets.client : < HTTP/1.1 101 WebSocket Protocol Handshake
DEBUG    : 2023-02-10 02:40:36,968 : websockets.client : < Upgrade: WebSocket
DEBUG    : 2023-02-10 02:40:36,968 : websockets.client : < Connection: Upgrade
DEBUG    : 2023-02-10 02:40:36,969 : websockets.client : < Sec-WebSocket-Accept: 1Vff/ccWM9WOTWKdZigUywH95zI=
DEBUG    : 2023-02-10 02:40:36,971 : websockets.client : < Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits=15
DEBUG    : 2023-02-10 02:40:36,972 : websockets.client : = connection is OPEN
DEBUG    : 2023-02-10 02:40:36,973 : websockets.client : > TEXT '{"id": 1, "method": "Target.setDiscoverTargets"...s": {"discover": true}}' [78 bytes]
DEBUG    : 2023-02-10 02:40:36,980 : websockets.client : < TEXT '{"method":"Target.targetCreated","params":{"tar...":"","attached":true}}}' [162 bytes]
DEBUG    : 2023-02-10 02:40:36,981 : websockets.client : < TEXT '{"method":"Target.targetCreated","params":{"tar...:"","attached":false}}}' [163 bytes]
DEBUG    : 2023-02-10 02:40:36,981 : websockets.client : < TEXT '{"method":"Target.targetCreated","params":{"tar...583B09AB22E3A6BCD11"}}}' [221 bytes]
DEBUG    : 2023-02-10 02:40:36,983 : websockets.client : < TEXT '{"id":1,"result":{}}' [20 bytes]
DEBUG    : 2023-02-10 02:40:36,984 : pyppeteer.connection.Connection : RECV: {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"28dc2d38-5e24-42c2-be8e-f584b819d067","type":"browser","title":"","url":"","attached":true}}}
DEBUG    : 2023-02-10 02:40:36,985 : pyppeteer.connection.Connection : RECV: {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"2e284177-55a4-4747-936f-8fcfce4daa84","type":"browser","title":"","url":"","attached":false}}}
DEBUG    : 2023-02-10 02:40:36,986 : pyppeteer.connection.Connection : RECV: {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"8FBB603CDE1315D86EB3E46D95B602D5","type":"page","title":"","url":"about:blank","attached":false,"browserContextId":"59BFB1FBAA9D2583B09AB22E3A6BCD11"}}}
DEBUG    : 2023-02-10 02:40:36,987 : pyppeteer.connection.Connection : RECV: {"id":1,"result":{}}
INFO     : 2023-02-10 02:40:36,988 : __main__ : start 'newPage()'
DEBUG    : 2023-02-10 02:40:36,989 : pyppeteer.connection.Connection : SEND: {"id": 2, "method": "Target.createTarget", "params": {"url": "about:blank"}}
DEBUG    : 2023-02-10 02:40:36,989 : websockets.client : > TEXT '{"id": 2, "method": "Target.createTarget", "par...{"url": "about:blank"}}' [76 bytes]
DEBUG    : 2023-02-10 02:40:36,998 : websockets.client : < TEXT '{"method":"Target.targetCreated","params":{"tar...583B09AB22E3A6BCD11"}}}' [210 bytes]
DEBUG    : 2023-02-10 02:40:37,000 : pyppeteer.connection.Connection : RECV: {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"341CA1FE1641430B96936CFA2DCA14FC","type":"page","title":"","url":"","attached":false,"browserContextId":"59BFB1FBAA9D2583B09AB22E3A6BCD11"}}}
DEBUG    : 2023-02-10 02:40:37,002 : websockets.client : < TEXT '{"id":2,"result":{"targetId":"341CA1FE1641430B96936CFA2DCA14FC"}}' [65 bytes]
DEBUG    : 2023-02-10 02:40:37,003 : pyppeteer.connection.Connection : RECV: {"id":2,"result":{"targetId":"341CA1FE1641430B96936CFA2DCA14FC"}}
DEBUG    : 2023-02-10 02:40:37,008 : websockets.client : < TEXT '{"method":"Target.targetInfoChanged","params":{...583B09AB22E3A6BCD11"}}}' [225 bytes]
DEBUG    : 2023-02-10 02:40:37,010 : websockets.client : < TEXT '{"method":"Target.targetCrashed","params":{"tar...ashed","errorCode":31}}' [124 bytes]
DEBUG    : 2023-02-10 02:40:37,011 : pyppeteer.connection.Connection : RECV: {"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"341CA1FE1641430B96936CFA2DCA14FC","type":"page","title":"","url":"about:blank","attached":false,"browserContextId":"59BFB1FBAA9D2583B09AB22E3A6BCD11"}}}
DEBUG    : 2023-02-10 02:40:37,013 : pyppeteer.connection.Connection : SEND: {"id": 3, "method": "Target.attachToTarget", "params": {"targetId": "341CA1FE1641430B96936CFA2DCA14FC"}}
DEBUG    : 2023-02-10 02:40:37,013 : pyppeteer.connection.Connection : RECV: {"method":"Target.targetCrashed","params":{"targetId":"341CA1FE1641430B96936CFA2DCA14FC","status":"crashed","errorCode":31}}
DEBUG    : 2023-02-10 02:40:37,015 : websockets.client : > TEXT '{"id": 3, "method": "Target.attachToTarget", "p...430B96936CFA2DCA14FC"}}' [104 bytes]
DEBUG    : 2023-02-10 02:40:37,016 : websockets.client : < TEXT '{"method":"Target.targetInfoChanged","params":{...583B09AB22E3A6BCD11"}}}' [224 bytes]
DEBUG    : 2023-02-10 02:40:37,017 : websockets.client : < TEXT '{"method":"Target.attachedToTarget","params":{"...ingForDebugger":false}}' [297 bytes]
DEBUG    : 2023-02-10 02:40:37,018 : websockets.client : < TEXT '{"id":3,"result":{"sessionId":"5760D9CD965DC546B37FADB72DDFB0F3"}}' [66 bytes]
DEBUG    : 2023-02-10 02:40:37,019 : pyppeteer.connection.Connection : RECV: {"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"341CA1FE1641430B96936CFA2DCA14FC","type":"page","title":"","url":"about:blank","attached":true,"browserContextId":"59BFB1FBAA9D2583B09AB22E3A6BCD11"}}}
DEBUG    : 2023-02-10 02:40:37,021 : pyppeteer.connection.Connection : RECV: {"method":"Target.attachedToTarget","params":{"sessionId":"5760D9CD965DC546B37FADB72DDFB0F3","targetInfo":{"targetId":"341CA1FE1641430B96936CFA2DCA14FC","type":"page","title":"","url":"about:blank","attached":true,"browserContextId":"59BFB1FBAA9D2583B09AB22E3A6BCD11"},"waitingForDebugger":false}}
DEBUG    : 2023-02-10 02:40:37,022 : pyppeteer.connection.Connection : RECV: {"id":3,"result":{"sessionId":"5760D9CD965DC546B37FADB72DDFB0F3"}}
DEBUG    : 2023-02-10 02:40:37,023 : pyppeteer.connection.CDPSession : SEND: {"id": 1, "method": "Page.enable", "params": null}
DEBUG    : 2023-02-10 02:40:37,024 : pyppeteer.connection.Connection : SEND: {"id": 4, "method": "Target.sendMessageToTarget", "params": {"sessionId": "5760D9CD965DC546B37FADB72DDFB0F3", "message": "{\"id\": 1, \"method\": \"Page.enable\", \"params\": null}"}}
DEBUG    : 2023-02-10 02:40:37,025 : websockets.client : > TEXT '{"id": 4, "method": "Target.sendMessageToTarget... \\"params\\": null}"}}' [183 bytes]
DEBUG    : 2023-02-10 02:40:37,027 : websockets.client : < TEXT '{"id":4,"result":{}}' [20 bytes]
DEBUG    : 2023-02-10 02:40:37,029 : pyppeteer.connection.Connection : RECV: {"id":4,"result":{}}

追記

kuzira5186さんの回答でpage.gotoawaitを付け忘れていたことに気付いたので、awaitを付けて再度実行ました。
しかし、、変らず処理が途中で止まり、pythonコマンドは終了しませんでした。

11

2 件の回答 2

0

page.gotoawait が必要ですね!別の問題ならすみません!

await page.goto("https://www.google.com/")
1
  • ありがとうございます。確かにawaitが抜けていました。ただ、awaitを付けても、結果は変わりませんでした。
    – yuji38kwmt
    Commented 2023年3月13日 1:53
0

metropolisさんのコメント通り、launch関数にargs=['--no-sandbox']を指定したら、期待通り動くようになりました。

import asyncio
from pyppeteer import launch
from pyppeteer.browser import Browser
from pyppeteer.page import Page

import logging

logger = logging.getLogger(__name__)

logging.basicConfig(format="%(levelname)-8s : %(asctime)s : %(name)s : %(message)s", level=logging.INFO)


async def main():
    logger.info("start 'launch()'")
    browser: Browser = await launch(args=["--no-sandbox"])
    logger.info("start 'newPage()'")
    page: Page = await browser.newPage()
    logger.info("start 'goto()'")
    await page.goto("https://www.google.com/")
    logger.info(f"title='{await page.title()}'")
    await browser.close()


if __name__ == "__main__":
    asyncio.new_event_loop().run_until_complete(main())
$ python sample.py
INFO     : 2023-03-14 08:09:35,321 : __main__ : start 'launch()'
INFO     : 2023-03-14 08:09:35,445 : pyppeteer.launcher : Browser listening on: ws://127.0.0.1:34889/devtools/browser/f12e7475-1b6d-481b-9cc6-55abbbc1ed8d
INFO     : 2023-03-14 08:09:35,449 : __main__ : start 'newPage()'
INFO     : 2023-03-14 08:09:35,535 : __main__ : start 'goto()'
INFO     : 2023-03-14 08:09:36,757 : __main__ : title='Google'
INFO     : 2023-03-14 08:09:36,758 : pyppeteer.launcher : terminate chrome process...

この質問に回答するには、ログインする必要があります。

求めていた回答ではありませんか? のタグが付いた他の質問を参照する。