Mon May 9 16:43:21 2022 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 470.57.02 Driver Version: 470.57.02 CUDA Version: 11.4 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Quadro RTX 8000 On | 00000000:2F:00.0 Off | 0 | | N/A 58C P8 36W / 250W | 0MiB / 45556MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+ Obtaining file:///scratch/rb5139/constellations/src Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Installing collected packages: constellations Attempting uninstall: constellations Found existing installation: constellations 0.0.0 Uninstalling constellations-0.0.0: Successfully uninstalled constellations-0.0.0 Running setup.py develop for constellations Successfully installed constellations-0.0.0 Creating .ZBU27M9AA9JEQX7Q5OHX, for loading in git data Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Creating .22ADG6UC9CPSDULOOPOO, for loading in git data Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Euclidean distance between 95 and 96: 21.227619171142578. Interpolating between 95 and 96 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2750 0.9199 0.8871 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2627 0.9160 0.8828 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2579 0.9102 0.8757 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2597 0.9141 0.8824 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2635 0.9141 0.8830 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2589 0.9121 0.8794 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2543 0.9102 0.8750 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2535 0.9102 0.8750 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2549 0.9102 0.8743 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2586 0.9180 0.8840 Completed interpolation from 95 to 96 Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Creating .M4QERO8KRKR82LDP8K92, for loading in git data Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Euclidean distance between 95 and 97: 22.05683708190918. Interpolating between 95 and 97 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2718 0.9043 0.8650 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2521 0.9062 0.8689 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2449 0.9121 0.8780 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2464 0.9043 0.8686 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2503 0.9062 0.8723 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2499 0.9082 0.8747 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2486 0.9082 0.8740 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2479 0.9121 0.8787 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2515 0.9121 0.8774 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2586 0.9180 0.8840 Completed interpolation from 95 to 97 Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Creating .K58Z3RBU7BYNWVHTW0RK, for loading in git data Encountered Error: connectivity/bert_ft_qqp-98 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-98.pt Euclidean distance between 95 and 98: 21.55661392211914. Interpolating between 95 and 98 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2615 0.9043 0.8643 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2454 0.9102 0.8736 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2377 0.9062 0.8696 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2365 0.9082 0.8733 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2385 0.9043 0.8686 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2371 0.9062 0.8710 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2366 0.9102 0.8763 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2397 0.9141 0.8811 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2473 0.9121 0.8774 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2586 0.9180 0.8840 Completed interpolation from 95 to 98 Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Encountered Error: connectivity/bert_ft_qqp-95 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-95.pt Creating .H3XY655L4662XWNJ8OW4, for loading in git data Encountered Error: connectivity/bert_ft_qqp-99 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-99.pt Euclidean distance between 95 and 99: 23.654468536376953. Interpolating between 95 and 99 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2525 0.9121 0.8747 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2205 0.9160 0.8815 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2124 0.9121 0.8787 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2256 0.9160 0.8847 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2396 0.9062 0.8750 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2367 0.9082 0.8779 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2279 0.9004 0.8661 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2287 0.9082 0.8747 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2399 0.9082 0.8726 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2586 0.9180 0.8840 Completed interpolation from 95 to 99 Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Euclidean distance between 96 and 97: 21.759183883666992. Interpolating between 96 and 97 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2718 0.9043 0.8650 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2492 0.9023 0.8626 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2397 0.9062 0.8696 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2407 0.9062 0.8703 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2450 0.9082 0.8733 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2472 0.9082 0.8733 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2496 0.9082 0.8733 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2544 0.9121 0.8780 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2621 0.9160 0.8828 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2750 0.9199 0.8871 Completed interpolation from 96 to 97 Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-98 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-98.pt Euclidean distance between 96 and 98: 21.64398193359375. Interpolating between 96 and 98 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2615 0.9043 0.8643 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2520 0.9082 0.8705 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2506 0.9082 0.8726 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2560 0.9062 0.8703 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2614 0.9062 0.8710 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2635 0.9102 0.8763 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2606 0.9102 0.8757 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2561 0.9102 0.8750 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2618 0.9160 0.8828 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2750 0.9199 0.8871 Completed interpolation from 96 to 98 Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-96 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-96.pt Encountered Error: connectivity/bert_ft_qqp-99 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-99.pt Euclidean distance between 96 and 99: 23.395904541015625. Interpolating between 96 and 99 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2525 0.9121 0.8747 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2166 0.9180 0.8846 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2077 0.9160 0.8835 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2256 0.9121 0.8819 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2477 0.9102 0.8814 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2501 0.8965 0.8638 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2390 0.9121 0.8819 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2365 0.9180 0.8877 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2499 0.9121 0.8780 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2750 0.9199 0.8871 Completed interpolation from 96 to 99 Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Encountered Error: connectivity/bert_ft_qqp-98 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-98.pt Euclidean distance between 97 and 98: 22.01187515258789. Interpolating between 97 and 98 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2615 0.9043 0.8643 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2491 0.9043 0.8658 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2436 0.9062 0.8696 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2435 0.9043 0.8679 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2454 0.9043 0.8686 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2462 0.9062 0.8710 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2445 0.9062 0.8703 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2445 0.9082 0.8733 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2521 0.9023 0.8634 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2718 0.9043 0.8650 Completed interpolation from 97 to 98 Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Encountered Error: connectivity/bert_ft_qqp-97 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-97.pt Encountered Error: connectivity/bert_ft_qqp-99 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-99.pt Euclidean distance between 97 and 99: 22.550342559814453. Interpolating between 97 and 99 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2525 0.9121 0.8747 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2255 0.9082 0.8691 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2143 0.9082 0.8705 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2168 0.9082 0.8712 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2236 0.9062 0.8703 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2264 0.9023 0.8649 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2249 0.9004 0.8625 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2301 0.8984 0.8587 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2452 0.9043 0.8650 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2718 0.9043 0.8650 Completed interpolation from 97 to 99 Encountered Error: connectivity/bert_ft_qqp-98 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-98.pt Encountered Error: connectivity/bert_ft_qqp-98 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-98.pt Encountered Error: connectivity/bert_ft_qqp-99 does not appear to have a file named flax_model.msgpack. Trying to load model from connectivity/bert_ft_qqp-99.pt Euclidean distance between 98 and 99: 23.562227249145508. Interpolating between 98 and 99 on qqp [0.0, 1.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 0 0.2525 0.9121 0.8747 [0.1111111111111111, 0.8888888888888888] point_num loss accuracy f1 ----------- -------- ---------- -------- 1 0.2209 0.9082 0.8691 [0.2222222222222222, 0.7777777777777778] point_num loss accuracy f1 ----------- -------- ---------- -------- 2 0.2165 0.9062 0.8674 [0.3333333333333333, 0.6666666666666667] point_num loss accuracy f1 ----------- -------- ---------- -------- 3 0.2397 0.9102 0.8743 [0.4444444444444444, 0.5555555555555556] point_num loss accuracy f1 ----------- -------- ---------- -------- 4 0.2687 0.9082 0.8719 [0.5555555555555556, 0.4444444444444444] point_num loss accuracy f1 ----------- -------- ---------- -------- 5 0.2624 0.9023 0.8649 [0.6666666666666666, 0.33333333333333337] point_num loss accuracy f1 ----------- -------- ---------- -------- 6 0.2375 0.9082 0.8747 [0.7777777777777777, 0.22222222222222232] point_num loss accuracy f1 ----------- -------- ---------- -------- 7 0.2306 0.9062 0.8717 [0.8888888888888888, 0.11111111111111116] point_num loss accuracy f1 ----------- -------- ---------- -------- 8 0.2399 0.9102 0.8757 [1.0, 0.0] point_num loss accuracy f1 ----------- -------- ---------- -------- 9 0.2615 0.9043 0.8643 Completed interpolation from 98 to 99 Wrote the values to ../logs/NLI/qqp_berts/qqp_interpol@34110steps/interpol_qqp_validation19.pkl!