Sun May 15 21:17:52 2022 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 470.57.02 Driver Version: 470.57.02 CUDA Version: 11.4 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Quadro RTX 8000 On | 00000000:2F:00.0 Off | 0 | | N/A 62C P8 34W / 250W | 0MiB / 45556MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+ Obtaining file:///scratch/rb5139/constellations/src Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Installing collected packages: constellations Attempting uninstall: constellations Found existing installation: constellations 0.0.0 Uninstalling constellations-0.0.0: Successfully uninstalled constellations-0.0.0 Running setup.py develop for constellations Successfully installed constellations-0.0.0 Creating .Z35PBWYN8YV4ZNWGS8XE, for loading in git data Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Creating .UQ71PN5U0WNBQW9AHN1G, for loading in git data Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Euclidean distance between 95 and 96: 20.95233726501465. Interpolating between 95 and 96 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2803 0.9004 0.8555 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2667 0.9043 0.8635 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2634 0.9121 0.8760 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2669 0.9141 0.8804 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2694 0.9121 0.8780 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2670 0.9102 0.8750 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2609 0.9082 0.8712 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2574 0.9062 0.8681 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2586 0.9082 0.8712 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2649 0.9043 0.8643 Completed interpolation from 95 to 96 Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Creating .JUJF1O54BBHC7LJWKC2M, for loading in git data Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Euclidean distance between 95 and 97: 21.752470016479492. Interpolating between 95 and 97 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2618 0.8984 0.8531 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2457 0.8965 0.8524 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2402 0.9043 0.8665 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2421 0.9023 0.8649 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2464 0.9043 0.8679 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2462 0.9023 0.8649 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2456 0.9082 0.8719 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2479 0.9062 0.8689 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2538 0.9082 0.8712 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2649 0.9043 0.8643 Completed interpolation from 95 to 97 Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Creating .J9RPBAZGWXAFZIHLO1U7, for loading in git data Encountered Error: connectivity/bert_ft_qqp-98 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-98.pt Euclidean distance between 95 and 98: 21.27008056640625. Interpolating between 95 and 98 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2600 0.9023 0.8588 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2449 0.9004 0.8571 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2380 0.8945 0.8500 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2378 0.8965 0.8540 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2408 0.9023 0.8634 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2402 0.9043 0.8658 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2403 0.9023 0.8634 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2439 0.9043 0.8658 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2520 0.9023 0.8619 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2649 0.9043 0.8643 Completed interpolation from 95 to 98 Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Creating .2YZYKSW84NQHCSJCADKA, for loading in git data Encountered Error: connectivity/bert_ft_qqp-99 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-99.pt Euclidean distance between 95 and 99: 23.2761287689209. Interpolating between 95 and 99 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2396 0.9062 0.8659 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2157 0.9082 0.8691 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2132 0.9043 0.8658 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2281 0.9082 0.8733 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2384 0.9160 0.8859 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2336 0.9141 0.8824 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2258 0.9102 0.8763 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2301 0.9062 0.8696 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2441 0.9062 0.8696 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2649 0.9043 0.8643 Completed interpolation from 95 to 99 Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Euclidean distance between 96 and 97: 21.468490600585938. Interpolating between 96 and 97 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2618 0.8984 0.8531 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2455 0.8945 0.8500 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2392 0.9023 0.8626 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2412 0.9043 0.8672 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2454 0.9023 0.8649 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2480 0.9023 0.8641 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2503 0.9043 0.8650 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2537 0.9082 0.8698 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2632 0.9082 0.8683 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2803 0.9004 0.8555 Completed interpolation from 96 to 97 Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-98 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-98.pt Euclidean distance between 96 and 98: 21.361433029174805. Interpolating between 96 and 98 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2600 0.9023 0.8588 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2554 0.9004 0.8563 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2576 0.8965 0.8515 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2656 0.8906 0.8444 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2747 0.8848 0.8347 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2755 0.8906 0.8436 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2701 0.8965 0.8524 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2652 0.9023 0.8603 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2679 0.9004 0.8571 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2803 0.9004 0.8555 Completed interpolation from 96 to 98 Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-99 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-99.pt Euclidean distance between 96 and 99: 23.032705307006836. Interpolating between 96 and 99 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2396 0.9062 0.8659 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2135 0.9062 0.8667 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2127 0.9004 0.8610 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2344 0.9141 0.8836 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2557 0.9043 0.8734 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2532 0.8965 0.8623 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2400 0.9062 0.8723 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2373 0.9102 0.8750 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2507 0.9062 0.8659 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2803 0.9004 0.8555 Completed interpolation from 96 to 99 Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Encountered Error: connectivity/bert_ft_qqp-98 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-98.pt Euclidean distance between 97 and 98: 21.71268653869629. Interpolating between 97 and 98 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2600 0.9023 0.8588 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2486 0.9004 0.8571 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2427 0.8945 0.8500 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2420 0.8945 0.8525 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2441 0.8945 0.8525 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2438 0.8984 0.8579 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2407 0.8965 0.8548 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2399 0.8945 0.8508 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2468 0.8926 0.8468 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2618 0.8984 0.8531 Completed interpolation from 97 to 98 Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Encountered Error: connectivity/bert_ft_qqp-99 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-99.pt Euclidean distance between 97 and 99: 22.212757110595703. Interpolating between 97 and 99 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2396 0.9062 0.8659 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2207 0.9023 0.8603 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2150 0.9043 0.8643 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2206 0.8945 0.8516 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2283 0.8965 0.8540 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2290 0.9004 0.8610 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2261 0.9004 0.8603 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2281 0.8945 0.8508 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2396 0.8906 0.8436 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2618 0.8984 0.8531 Completed interpolation from 97 to 99 Encountered Error: connectivity/bert_ft_qqp-98 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-98.pt Encountered Error: connectivity/bert_ft_qqp-98 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-98.pt Encountered Error: connectivity/bert_ft_qqp-99 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-99.pt Euclidean distance between 98 and 99: 23.194971084594727. Interpolating between 98 and 99 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2396 0.9062 0.8659 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2183 0.9082 0.8669 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2240 0.9043 0.8596 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2558 0.8926 0.8442 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2873 0.8867 0.8362 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2772 0.8906 0.8418 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2475 0.8965 0.8532 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2351 0.8945 0.8500 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2416 0.8965 0.8515 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2600 0.9023 0.8588 Completed interpolation from 98 to 99 Wrote the values to ../logs/NLI/qqp_berts/qqp_interpol@25000steps/interpol_qqp_validation19.pkl!