Sun May 15 14:08:15 2022 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 470.57.02 Driver Version: 470.57.02 CUDA Version: 11.4 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Quadro RTX 8000 On | 00000000:86:00.0 Off | 0 | | N/A 26C P8 13W / 250W | 0MiB / 45556MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+ Obtaining file:///scratch/rb5139/constellations/src Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Installing collected packages: constellations Attempting uninstall: constellations Found existing installation: constellations 0.0.0 Uninstalling constellations-0.0.0: Successfully uninstalled constellations-0.0.0 Running setup.py develop for constellations Successfully installed constellations-0.0.0 Creating .16MB9L5NLTNNO4QPJACL, for loading in git data Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Creating .7PIFW8RDK8XC03JY1VH1, for loading in git data Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Euclidean distance between 95 and 96: 18.774574279785156. Interpolating between 95 and 96 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2561 0.8945 0.8448 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2484 0.9004 0.8563 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2476 0.9121 0.8760 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2538 0.9062 0.8696 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2608 0.9062 0.8696 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2575 0.9062 0.8696 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2499 0.9023 0.8619 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2452 0.9004 0.8595 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2443 0.9023 0.8603 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2482 0.9062 0.8644 Completed interpolation from 95 to 96 Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Creating .C0VPFTNWXB2T78080HD5, for loading in git data Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Euclidean distance between 95 and 97: 19.503650665283203. Interpolating between 95 and 97 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2330 0.8984 0.8547 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2306 0.9102 0.8743 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2359 0.9082 0.8719 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2448 0.9062 0.8703 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2491 0.8984 0.8610 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2460 0.8965 0.8579 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2417 0.9004 0.8633 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2396 0.8984 0.8571 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2410 0.9004 0.8579 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2482 0.9062 0.8644 Completed interpolation from 95 to 97 Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Creating .ZTHIZRW7BI40A12MP24L, for loading in git data Encountered Error: connectivity/bert_ft_qqp-98 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-98.pt Euclidean distance between 95 and 98: 19.072668075561523. Interpolating between 95 and 98 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2532 0.8965 0.8464 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2395 0.9004 0.8547 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2359 0.9102 0.8722 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2402 0.9043 0.8672 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2444 0.9023 0.8656 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2421 0.9062 0.8703 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2389 0.9043 0.8658 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2372 0.8984 0.8564 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2394 0.9043 0.8627 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2482 0.9062 0.8644 Completed interpolation from 95 to 98 Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Creating .A6IVVJA8UD7RHGWVF3IE, for loading in git data Encountered Error: connectivity/bert_ft_qqp-99 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-99.pt Euclidean distance between 95 and 99: 20.792173385620117. Interpolating between 95 and 99 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2399 0.8926 0.8424 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2247 0.8926 0.8451 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2326 0.9023 0.8641 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2543 0.9004 0.8668 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2682 0.8965 0.8651 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2583 0.8984 0.8660 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2403 0.9023 0.8691 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2309 0.8965 0.8556 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2327 0.9023 0.8611 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2482 0.9062 0.8644 Completed interpolation from 95 to 99 Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Euclidean distance between 96 and 97: 19.26505470275879. Interpolating between 96 and 97 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2330 0.8984 0.8547 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2295 0.9102 0.8729 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2337 0.9062 0.8689 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2426 0.9043 0.8665 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2475 0.9043 0.8672 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2446 0.9043 0.8665 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2391 0.9043 0.8658 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2380 0.9043 0.8650 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2442 0.9023 0.8596 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2561 0.8945 0.8448 Completed interpolation from 96 to 97 Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-98 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-98.pt Euclidean distance between 96 and 98: 19.161643981933594. Interpolating between 96 and 98 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2532 0.8965 0.8464 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2464 0.8984 0.8523 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2456 0.9004 0.8555 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2521 0.9043 0.8635 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2606 0.9023 0.8611 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2616 0.9043 0.8635 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2544 0.9043 0.8635 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2474 0.9043 0.8627 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2483 0.8965 0.8499 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2561 0.8945 0.8448 Completed interpolation from 96 to 98 Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-99 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-99.pt Euclidean distance between 96 and 99: 20.5821475982666. Interpolating between 96 and 99 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2399 0.8926 0.8424 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2254 0.8926 0.8451 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2359 0.9043 0.8679 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2682 0.8945 0.8608 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2982 0.8887 0.8564 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2908 0.8867 0.8550 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2603 0.8926 0.8586 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2385 0.9043 0.8686 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2380 0.9043 0.8650 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2561 0.8945 0.8448 Completed interpolation from 96 to 99 Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Encountered Error: connectivity/bert_ft_qqp-98 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-98.pt Euclidean distance between 97 and 98: 19.476078033447266. Interpolating between 97 and 98 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2532 0.8965 0.8464 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2392 0.8984 0.8531 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2354 0.9023 0.8596 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2404 0.9023 0.8634 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2495 0.9004 0.8625 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2523 0.9004 0.8618 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2444 0.8984 0.8587 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2343 0.9023 0.8634 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2296 0.9062 0.8674 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2330 0.8984 0.8547 Completed interpolation from 97 to 98 Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Encountered Error: connectivity/bert_ft_qqp-99 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-99.pt Euclidean distance between 97 and 99: 19.843801498413086. Interpolating between 97 and 99 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2399 0.8926 0.8424 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2275 0.8965 0.8499 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2277 0.9043 0.8643 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2368 0.8965 0.8548 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2451 0.8945 0.8548 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2459 0.9004 0.8633 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2372 0.9004 0.8625 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2284 0.9004 0.8618 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2260 0.9043 0.8658 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2330 0.8984 0.8547 Completed interpolation from 97 to 99 Encountered Error: connectivity/bert_ft_qqp-98 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-98.pt Encountered Error: connectivity/bert_ft_qqp-98 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-98.pt Encountered Error: connectivity/bert_ft_qqp-99 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-99.pt Euclidean distance between 98 and 99: 20.736886978149414. Interpolating between 98 and 99 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2399 0.8926 0.8424 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2303 0.8926 0.8424 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2456 0.8926 0.8433 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2832 0.8965 0.8490 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.3186 0.8965 0.8507 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.3067 0.8945 0.8500 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2635 0.8965 0.8532 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2376 0.9004 0.8579 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2363 0.8965 0.8499 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2532 0.8965 0.8464 Completed interpolation from 98 to 99 Wrote the values to ../logs/NLI/qqp_berts/qqp_interpol@15000steps/interpol_qqp_validation19.pkl!