環境
- pyppeteer 1.0.2
- Python 3.10.2
- Ubuntu 22.04
- VirtualBox v7.0.6 r155176を利用している
- ホストOSはWindows11 Pro
問題点
pyppeteerでブラウザを操作したいです。
以下のコードを実行したところ、page.goto
を実行する前で、処理が止まりました。
質問
下記のコードが途中で処理が止まってしまう原因と解決策を教えていただきたいです。
以下に記載のログで、最終行のRECV: {"id":4,"result":{}}
以降は、1分以上待ってもログは出力されませんでした。
補足
先日Ubuntu18.04から22.04にバージョンアップしました。
Ubuntu18.04のときは上記のようなコード(厳密に一緒ではないが)は動きました。
Ubuntuのバージョンアップにより、何かが変わったのでしょうか?
現状のコード:
import os
import asyncio
from pyppeteer import launch
from pyppeteer.browser import Browser
from pyppeteer.page import Page
from typing import Optional
import time
import logging
logger = logging.getLogger(__name__)
logging.basicConfig(format='%(levelname)-8s : %(asctime)s : %(name)s : %(message)s', level=logging.DEBUG)
async def main():
logger.info("start 'launch()'")
browser: Browser = await launch()
logger.info("start 'newPage()'")
page: Page = await browser.newPage()
logger.info("start 'goto()'")
page.goto("https://www.google.com/")
logger.info(f"title='{page.title}'")
await browser.close()
if __name__ == "__main__":
asyncio.new_event_loop().run_until_complete(main())
ログ:
$ python sample.py
DEBUG : 2023-02-10 02:40:36,722 : asyncio : Using selector: EpollSelector
INFO : 2023-02-10 02:40:36,723 : __main__ : start 'launch()'
INFO : 2023-02-10 02:40:36,962 : pyppeteer.launcher : Browser listening on: ws://127.0.0.1:44795/devtools/browser/41f9db60-def6-4e84-86bb-47a8238e1269
DEBUG : 2023-02-10 02:40:36,962 : pyppeteer.connection.Connection : SEND: {"id": 1, "method": "Target.setDiscoverTargets", "params": {"discover": true}}
DEBUG : 2023-02-10 02:40:36,963 : websockets.client : = connection is CONNECTING
DEBUG : 2023-02-10 02:40:36,963 : websockets.client : > GET /devtools/browser/41f9db60-def6-4e84-86bb-47a8238e1269 HTTP/1.1
DEBUG : 2023-02-10 02:40:36,963 : websockets.client : > Host: 127.0.0.1:44795
DEBUG : 2023-02-10 02:40:36,964 : websockets.client : > Upgrade: websocket
DEBUG : 2023-02-10 02:40:36,964 : websockets.client : > Connection: Upgrade
DEBUG : 2023-02-10 02:40:36,964 : websockets.client : > Sec-WebSocket-Key: x249CRPQgAIEzSuRpwfwWA==
DEBUG : 2023-02-10 02:40:36,964 : websockets.client : > Sec-WebSocket-Version: 13
DEBUG : 2023-02-10 02:40:36,964 : websockets.client : > Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
DEBUG : 2023-02-10 02:40:36,964 : websockets.client : > User-Agent: Python/3.10 websockets/10.4
DEBUG : 2023-02-10 02:40:36,968 : websockets.client : < HTTP/1.1 101 WebSocket Protocol Handshake
DEBUG : 2023-02-10 02:40:36,968 : websockets.client : < Upgrade: WebSocket
DEBUG : 2023-02-10 02:40:36,968 : websockets.client : < Connection: Upgrade
DEBUG : 2023-02-10 02:40:36,969 : websockets.client : < Sec-WebSocket-Accept: 1Vff/ccWM9WOTWKdZigUywH95zI=
DEBUG : 2023-02-10 02:40:36,971 : websockets.client : < Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits=15
DEBUG : 2023-02-10 02:40:36,972 : websockets.client : = connection is OPEN
DEBUG : 2023-02-10 02:40:36,973 : websockets.client : > TEXT '{"id": 1, "method": "Target.setDiscoverTargets"...s": {"discover": true}}' [78 bytes]
DEBUG : 2023-02-10 02:40:36,980 : websockets.client : < TEXT '{"method":"Target.targetCreated","params":{"tar...":"","attached":true}}}' [162 bytes]
DEBUG : 2023-02-10 02:40:36,981 : websockets.client : < TEXT '{"method":"Target.targetCreated","params":{"tar...:"","attached":false}}}' [163 bytes]
DEBUG : 2023-02-10 02:40:36,981 : websockets.client : < TEXT '{"method":"Target.targetCreated","params":{"tar...583B09AB22E3A6BCD11"}}}' [221 bytes]
DEBUG : 2023-02-10 02:40:36,983 : websockets.client : < TEXT '{"id":1,"result":{}}' [20 bytes]
DEBUG : 2023-02-10 02:40:36,984 : pyppeteer.connection.Connection : RECV: {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"28dc2d38-5e24-42c2-be8e-f584b819d067","type":"browser","title":"","url":"","attached":true}}}
DEBUG : 2023-02-10 02:40:36,985 : pyppeteer.connection.Connection : RECV: {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"2e284177-55a4-4747-936f-8fcfce4daa84","type":"browser","title":"","url":"","attached":false}}}
DEBUG : 2023-02-10 02:40:36,986 : pyppeteer.connection.Connection : RECV: {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"8FBB603CDE1315D86EB3E46D95B602D5","type":"page","title":"","url":"about:blank","attached":false,"browserContextId":"59BFB1FBAA9D2583B09AB22E3A6BCD11"}}}
DEBUG : 2023-02-10 02:40:36,987 : pyppeteer.connection.Connection : RECV: {"id":1,"result":{}}
INFO : 2023-02-10 02:40:36,988 : __main__ : start 'newPage()'
DEBUG : 2023-02-10 02:40:36,989 : pyppeteer.connection.Connection : SEND: {"id": 2, "method": "Target.createTarget", "params": {"url": "about:blank"}}
DEBUG : 2023-02-10 02:40:36,989 : websockets.client : > TEXT '{"id": 2, "method": "Target.createTarget", "par...{"url": "about:blank"}}' [76 bytes]
DEBUG : 2023-02-10 02:40:36,998 : websockets.client : < TEXT '{"method":"Target.targetCreated","params":{"tar...583B09AB22E3A6BCD11"}}}' [210 bytes]
DEBUG : 2023-02-10 02:40:37,000 : pyppeteer.connection.Connection : RECV: {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"341CA1FE1641430B96936CFA2DCA14FC","type":"page","title":"","url":"","attached":false,"browserContextId":"59BFB1FBAA9D2583B09AB22E3A6BCD11"}}}
DEBUG : 2023-02-10 02:40:37,002 : websockets.client : < TEXT '{"id":2,"result":{"targetId":"341CA1FE1641430B96936CFA2DCA14FC"}}' [65 bytes]
DEBUG : 2023-02-10 02:40:37,003 : pyppeteer.connection.Connection : RECV: {"id":2,"result":{"targetId":"341CA1FE1641430B96936CFA2DCA14FC"}}
DEBUG : 2023-02-10 02:40:37,008 : websockets.client : < TEXT '{"method":"Target.targetInfoChanged","params":{...583B09AB22E3A6BCD11"}}}' [225 bytes]
DEBUG : 2023-02-10 02:40:37,010 : websockets.client : < TEXT '{"method":"Target.targetCrashed","params":{"tar...ashed","errorCode":31}}' [124 bytes]
DEBUG : 2023-02-10 02:40:37,011 : pyppeteer.connection.Connection : RECV: {"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"341CA1FE1641430B96936CFA2DCA14FC","type":"page","title":"","url":"about:blank","attached":false,"browserContextId":"59BFB1FBAA9D2583B09AB22E3A6BCD11"}}}
DEBUG : 2023-02-10 02:40:37,013 : pyppeteer.connection.Connection : SEND: {"id": 3, "method": "Target.attachToTarget", "params": {"targetId": "341CA1FE1641430B96936CFA2DCA14FC"}}
DEBUG : 2023-02-10 02:40:37,013 : pyppeteer.connection.Connection : RECV: {"method":"Target.targetCrashed","params":{"targetId":"341CA1FE1641430B96936CFA2DCA14FC","status":"crashed","errorCode":31}}
DEBUG : 2023-02-10 02:40:37,015 : websockets.client : > TEXT '{"id": 3, "method": "Target.attachToTarget", "p...430B96936CFA2DCA14FC"}}' [104 bytes]
DEBUG : 2023-02-10 02:40:37,016 : websockets.client : < TEXT '{"method":"Target.targetInfoChanged","params":{...583B09AB22E3A6BCD11"}}}' [224 bytes]
DEBUG : 2023-02-10 02:40:37,017 : websockets.client : < TEXT '{"method":"Target.attachedToTarget","params":{"...ingForDebugger":false}}' [297 bytes]
DEBUG : 2023-02-10 02:40:37,018 : websockets.client : < TEXT '{"id":3,"result":{"sessionId":"5760D9CD965DC546B37FADB72DDFB0F3"}}' [66 bytes]
DEBUG : 2023-02-10 02:40:37,019 : pyppeteer.connection.Connection : RECV: {"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"341CA1FE1641430B96936CFA2DCA14FC","type":"page","title":"","url":"about:blank","attached":true,"browserContextId":"59BFB1FBAA9D2583B09AB22E3A6BCD11"}}}
DEBUG : 2023-02-10 02:40:37,021 : pyppeteer.connection.Connection : RECV: {"method":"Target.attachedToTarget","params":{"sessionId":"5760D9CD965DC546B37FADB72DDFB0F3","targetInfo":{"targetId":"341CA1FE1641430B96936CFA2DCA14FC","type":"page","title":"","url":"about:blank","attached":true,"browserContextId":"59BFB1FBAA9D2583B09AB22E3A6BCD11"},"waitingForDebugger":false}}
DEBUG : 2023-02-10 02:40:37,022 : pyppeteer.connection.Connection : RECV: {"id":3,"result":{"sessionId":"5760D9CD965DC546B37FADB72DDFB0F3"}}
DEBUG : 2023-02-10 02:40:37,023 : pyppeteer.connection.CDPSession : SEND: {"id": 1, "method": "Page.enable", "params": null}
DEBUG : 2023-02-10 02:40:37,024 : pyppeteer.connection.Connection : SEND: {"id": 4, "method": "Target.sendMessageToTarget", "params": {"sessionId": "5760D9CD965DC546B37FADB72DDFB0F3", "message": "{\"id\": 1, \"method\": \"Page.enable\", \"params\": null}"}}
DEBUG : 2023-02-10 02:40:37,025 : websockets.client : > TEXT '{"id": 4, "method": "Target.sendMessageToTarget... \\"params\\": null}"}}' [183 bytes]
DEBUG : 2023-02-10 02:40:37,027 : websockets.client : < TEXT '{"id":4,"result":{}}' [20 bytes]
DEBUG : 2023-02-10 02:40:37,029 : pyppeteer.connection.Connection : RECV: {"id":4,"result":{}}
追記
kuzira5186さんの回答でpage.goto
にawait
を付け忘れていたことに気付いたので、await
を付けて再度実行ました。
しかし、、変らず処理が途中で止まり、pythonコマンドは終了しませんでした。
await
が付いているので、少なくともその辺の変更を試してみてはどうでしょう? pyppeteer-チュートリアル, 【Techの道も一歩から】第45回「pyppeteerを使いヘッドレスブラウザでログインして情報を取得する」launch
で--no-sandbox
を指定すると問題なく実行されます。browser: Browser = await launch(args=['--no-sandbox'])
また、page.goto()
にもawait
が必要です。page.title
は attribute value ではなく method なのでlogger.info(f"title='{await page.title()}'")
とする必要があります。