Google Colab Pro+上のGPUマシンでYOLOv5を用いた学習を行っています。
友人とjupyter notebookを共有したところ、こちらと友人側とで得られる結果が異なり困っています。
環境:Google Colab Pro+
仮想マシンのGPU:Tesla P100
%cd /path/to/YOLOv5/
!python train.py --img $IMGSIZE --batch $BATCHSIZE --epochs $EPOCHNUMBER\
--data /path/to/data.yaml --weights yolov5s6.pt\
--hyp /path/to/hyp.yaml --workers $WORKERNUMBER
ログ
Downloading https://ultralytics.com/assets/Arial.ttf to /root/.config/Ultralytics/Arial.ttf...
train: weights=yolov5s6.pt, cfg=, data=/path/to/data.yaml, hyp=/path/to/hyp.yaml, epochs=$EPOCHNUMBER, batch_size=$BATCHSIZE, imgsz=$IMAGESIZE, rect=False, resume=False, nosave=False, noval=False, noautoanchor=False, evolve=None, bucket=, cache=None, image_weights=False, device=, multi_scale=False, single_cls=False, optimizer=SGD, sync_bn=False, workers=$WORKERNUMBER, project=runs/train, name=exp, exist_ok=False, quad=False, linear_lr=False, label_smoothing=0.0, patience=100, freeze=[0], save_period=-1, local_rank=-1, entity=None, upload_dataset=False, bbox_interval=-1, artifact_alias=latest
github: Command 'git fetch && git config --get remote.origin.url' timed out after 5 seconds
YOLOv5 🚀 2022-1-7 torch 1.10.0+cu111 CUDA:0 (Tesla P100-PCIE-16GB, 16281MiB)
hyperparameters: lr0=0.01, lrf=0.1, momentum=0.937, weight_decay=0.0005, warmup_epochs=3.0, warmup_momentum=0.8, warmup_bias_lr=0.1, box=0.05, cls=0.5, cls_pw=1.0, obj=1.0, obj_pw=1.0, iou_t=0.2, anchor_t=4.0, fl_gamma=0.0, hsv_h=0.015, hsv_s=0.7, hsv_v=0.4, degrees=0.0, translate=0.1, scale=0.5, shear=0.0, perspective=0.0, flipud=0.5, fliplr=0.5, mosaic=1.0, mixup=0.5, copy_paste=0.0
Weights & Biases: run 'pip install wandb' to automatically track and visualize YOLOv5 🚀 runs (RECOMMENDED)
TensorBoard: Start with 'tensorboard --logdir runs/train', view at http://localhost:6006/
Overriding model.yaml nc=80 with nc=1
from n params module arguments
0 -1 1 3520 models.common.Conv [3, 32, 6, 2, 2]
1 -1 1 18560 models.common.Conv [32, 64, 3, 2]
2 -1 1 18816 models.common.C3 [64, 64, 1]
3 -1 1 73984 models.common.Conv [64, 128, 3, 2]
4 -1 2 115712 models.common.C3 [128, 128, 2]
5 -1 1 295424 models.common.Conv [128, 256, 3, 2]
6 -1 3 625152 models.common.C3 [256, 256, 3]
7 -1 1 885504 models.common.Conv [256, 384, 3, 2]
8 -1 1 665856 models.common.C3 [384, 384, 1]
9 -1 1 1770496 models.common.Conv [384, 512, 3, 2]
10 -1 1 1182720 models.common.C3 [512, 512, 1]
11 -1 1 656896 models.common.SPPF [512, 512, 5]
12 -1 1 197376 models.common.Conv [512, 384, 1, 1]
13 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
14 [-1, 8] 1 0 models.common.Concat [1]
15 -1 1 813312 models.common.C3 [768, 384, 1, False]
16 -1 1 98816 models.common.Conv [384, 256, 1, 1]
17 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
18 [-1, 6] 1 0 models.common.Concat [1]
19 -1 1 361984 models.common.C3 [512, 256, 1, False]
20 -1 1 33024 models.common.Conv [256, 128, 1, 1]
21 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
22 [-1, 4] 1 0 models.common.Concat [1]
23 -1 1 90880 models.common.C3 [256, 128, 1, False]
24 -1 1 147712 models.common.Conv [128, 128, 3, 2]
25 [-1, 20] 1 0 models.common.Concat [1]
26 -1 1 296448 models.common.C3 [256, 256, 1, False]
27 -1 1 590336 models.common.Conv [256, 256, 3, 2]
28 [-1, 16] 1 0 models.common.Concat [1]
29 -1 1 715008 models.common.C3 [512, 384, 1, False]
30 -1 1 1327872 models.common.Conv [384, 384, 3, 2]
31 [-1, 12] 1 0 models.common.Concat [1]
32 -1 1 1313792 models.common.C3 [768, 512, 1, False]
33 [23, 26, 29, 32] 1 23112 models.yolo.Detect [1, [[19, 27, 44, 40, 38, 94], [96, 68, 86, 152, 180, 137], [140, 301, 303, 264, 238, 542], [436, 615, 739, 380, 925, 792]], [128, 256, 384, 512]]
Model Summary: 355 layers, 12322312 parameters, 12322312 gradients, 16.2 GFLOPs
AutoAnchor: 6.95 anchors/target, 1.000 Best Possible Recall (BPR). Current anchors are a good fit to dataset ✅
Image sizes $IMAGESIZE train, $IMAGESIZE val
Using $WORKERNUMBER dataloader workers
Logging results to runs/train/exp1
Starting training for $EPOCHNUMBER epochs...
上記のコードおよびデータを共有し、こちらと友人側で実行したところ、
- こちら:53min/epoch, best.ptのファイルサイズが100MBほど
- 友人側:33min/epoch, best.ptのファイルサイズが 28MBほど
と実行内容や得られる結果に差が出てしまいます。
用いているpretrained fileは同じで、ログを見ても学習が始まるところまでは全く同じです。
どうして同じデータ、同じコードで違いが生じてしまうのでしょうか?
アドバイスをいただけますと幸いです。よろしくお願い致します。