1. 0.18 s $ /nix/store/9ypz3flqsrl5xl495mm8h645gadjsxi1-coreutils-9.11/bin/timeout --kill-after=15s 7200s /nix/store/23zk8sgzamrbmj1fday15szf7f2279s8-nix-2.34.7/bin/nix --extra-experimental-features nix-command --extra-experimental-features flakes flake clone git+https://github.com/NorfairKing/dspy --dest /tmp/nix-ci-worker/stable-pwd/repo/
  2. 0.20 s Cloning into '/tmp/nix-ci-worker/stable-pwd/repo'...
  3. 0.58 s remote: Enumerating objects: 54719, done.
  4. 0.61 s remote: Counting objects: 1% (1/78) remote: Counting objects: 2% (2/78) remote: Counting objects: 3% (3/78) remote: Counting objects: 5% (4/78) remote: Counting objects: 6% (5/78) remote: Counting objects: 7% (6/78) remote: Counting objects: 8% (7/78) remote: Counting objects: 10% (8/78) remote: Counting objects: 11% (9/78) remote: Counting objects: 12% (10/78) remote: Counting objects: 14% (11/78) remote: Counting objects: 15% (12/78) remote: Counting objects: 16% (13/78) remote: Counting objects: 17% (14/78) remote: Counting objects: 19% (15/78) remote: Counting objects: 20% (16/78) remote: Counting objects: 21% (17/78) remote: Counting objects: 23% (18/78) remote: Counting objects: 24% (19/78) remote: Counting objects: 25% (20/78) remote: Counting objects: 26% (21/78) remote: Counting objects: 28% (22/78) remote: Counting objects: 29% (23/78) remote: Counting objects: 30% (24/78) remote: Counting objects: 32% (25/78) remote: Counting objects: 33% (26/78) remote: Counting objects: 34% (27/78) remote: Counting objects: 35% (28/78) remote: Counting objects: 37% (29/78) remote: Counting objects: 38% (30/78) remote: Counting objects: 39% (31/78) remote: Counting objects: 41% (32/78) remote: Counting objects: 42% (33/78) remote: Counting objects: 43% (34/78) remote: Counting objects: 44% (35/78) remote: Counting objects: 46% (36/78) remote: Counting objects: 47% (37/78) remote: Counting objects: 48% (38/78) remote: Counting objects: 50% (39/78) remote: Counting objects: 51% (40/78) remote: Counting objects: 52% (41/78) remote: Counting objects: 53% (42/78) remote: Counting objects: 55% (43/78) remote: Counting objects: 56% (44/78) remote: Counting objects: 57% (45/78) remote: Counting objects: 58% (46/78) remote: Counting objects: 60% (47/78) remote: Counting objects: 61% (48/78) remote: Counting objects: 62% (49/78) remote: Counting objects: 64% (50/78) remote: Counting objects: 65% (51/78) remote: Counting objects: 66% (52/78) remote: Counting objects: 67% (53/78) remote: Counting objects: 69% (54/78) remote: Counting objects: 70% (55/78) remote: Counting objects: 71% (56/78) remote: Counting objects: 73% (57/78) remote: Counting objects: 74% (58/78) remote: Counting objects: 75% (59/78) remote: Counting objects: 76% (60/78) remote: Counting objects: 78% (61/78) remote: Counting objects: 79% (62/78) remote: Counting objects: 80% (63/78) remote: Counting objects: 82% (64/78) remote: Counting objects: 83% (65/78) remote: Counting objects: 84% (66/78) remote: Counting objects: 85% (67/78) remote: Counting objects: 87% (68/78) remote: Counting objects: 88% (69/78) remote: Counting objects: 89% (70/78) remote: Counting objects: 91% (71/78) remote: Counting objects: 92% (72/78) remote: Counting objects: 93% (73/78) remote: Counting objects: 94% (74/78) remote: Counting objects: 96% (75/78) remote: Counting objects: 97% (76/78) remote: Counting objects: 98% (77/78) remote: Counting objects: 100% (78/78) remote: Counting objects: 100% (78/78), done.
  5. 0.62 s remote: Compressing objects: 2% (1/50) remote: Compressing objects: 4% (2/50) remote: Compressing objects: 6% (3/50) remote: Compressing objects: 8% (4/50) remote: Compressing objects: 10% (5/50) remote: Compressing objects: 12% (6/50) remote: Compressing objects: 14% (7/50) remote: Compressing objects: 16% (8/50) remote: Compressing objects: 18% (9/50) remote: Compressing objects: 20% (10/50) remote: Compressing objects: 22% (11/50) remote: Compressing objects: 24% (12/50) remote: Compressing objects: 26% (13/50) remote: Compressing objects: 28% (14/50) remote: Compressing objects: 30% (15/50) remote: Compressing objects: 32% (16/50) remote: Compressing objects: 34% (17/50) remote: Compressing objects: 36% (18/50) remote: Compressing objects: 38% (19/50) remote: Compressing objects: 40% (20/50) remote: Compressing objects: 42% (21/50) remote: Compressing objects: 44% (22/50) remote: Compressing objects: 46% (23/50) remote: Compressing objects: 48% (24/50) remote: Compressing objects: 50% (25/50) remote: Compressing objects: 52% (26/50) remote: Compressing objects: 54% (27/50) remote: Compressing objects: 56% (28/50) remote: Compressing objects: 58% (29/50) remote: Compressing objects: 60% (30/50) remote: Compressing objects: 62% (31/50) remote: Compressing objects: 64% (32/50) remote: Compressing objects: 66% (33/50) remote: Compressing objects: 68% (34/50) remote: Compressing objects: 70% (35/50) remote: Compressing objects: 72% (36/50) remote: Compressing objects: 74% (37/50) remote: Compressing objects: 76% (38/50) remote: Compressing objects: 78% (39/50) remote: Compressing objects: 80% (40/50) remote: Compressing objects: 82% (41/50) remote: Compressing objects: 84% (42/50) remote: Compressing objects: 86% (43/50) remote: Compressing objects: 88% (44/50) remote: Compressing objects: 90% (45/50) remote: Compressing objects: 92% (46/50) remote: Compressing objects: 94% (47/50) remote: Compressing objects: 96% (48/50) remote: Compressing objects: 98% (49/50) remote: Compressing objects: 100% (50/50) remote: Compressing objects: 100% (50/50), done.
  6. 5.48 s Receiving objects: 0% (1/54719) Receiving objects: 1% (548/54719) Receiving objects: 2% (1095/54719) Receiving objects: 3% (1642/54719) Receiving objects: 4% (2189/54719) Receiving objects: 5% (2736/54719) Receiving objects: 6% (3284/54719) Receiving objects: 7% (3831/54719) Receiving objects: 8% (4378/54719) Receiving objects: 9% (4925/54719), 17.47 MiB | 33.18 MiB/s Receiving objects: 10% (5472/54719), 17.47 MiB | 33.18 MiB/s Receiving objects: 11% (6020/54719), 17.47 MiB | 33.18 MiB/s Receiving objects: 12% (6567/54719), 17.47 MiB | 33.18 MiB/s Receiving objects: 13% (7114/54719), 17.47 MiB | 33.18 MiB/s Receiving objects: 13% (7545/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 14% (7661/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 15% (8208/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 16% (8756/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 17% (9303/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 18% (9850/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 19% (10397/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 20% (10944/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 21% (11491/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 22% (12039/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 23% (12586/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 24% (13133/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 25% (13680/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 26% (14227/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 27% (14775/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 28% (15322/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 29% (15869/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 30% (16416/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 31% (16963/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 32% (17511/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 33% (18058/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 34% (18605/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 35% (19152/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 36% (19699/54719), 51.71 MiB | 33.77 MiB/s Receiving objects: 37% (20247/54719), 51.71 MiB | 33.77 MiB/s Receiving objects: 38% (20794/54719), 51.71 MiB | 33.77 MiB/s Receiving objects: 39% (21341/54719), 51.71 MiB | 33.77 MiB/s Receiving objects: 39% (21872/54719), 69.08 MiB | 34.00 MiB/s Receiving objects: 40% (21888/54719), 69.08 MiB | 34.00 MiB/s Receiving objects: 41% (22435/54719), 69.08 MiB | 34.00 MiB/s Receiving objects: 42% (22982/54719), 69.08 MiB | 34.00 MiB/s Receiving objects: 43% (23530/54719), 69.08 MiB | 34.00 MiB/s Receiving objects: 44% (24077/54719), 87.01 MiB | 34.31 MiB/s Receiving objects: 45% (24624/54719), 87.01 MiB | 34.31 MiB/s Receiving objects: 46% (25171/54719), 87.01 MiB | 34.31 MiB/s Receiving objects: 47% (25718/54719), 87.01 MiB | 34.31 MiB/s Receiving objects: 48% (26266/54719), 87.01 MiB | 34.31 MiB/s Receiving objects: 48% (26400/54719), 104.09 MiB | 34.28 MiB/s Receiving objects: 49% (26813/54719), 121.48 MiB | 34.35 MiB/s Receiving objects: 49% (27101/54719), 121.48 MiB | 34.35 MiB/s Receiving objects: 50% (27360/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 51% (27907/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 52% (28454/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 53% (29002/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 54% (29549/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 55% (30096/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 56% (30643/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 57% (31190/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 58% (31738/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 59% (32285/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 60% (32832/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 61% (33379/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 62% (33926/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 63% (34473/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 64% (35021/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 65% (35568/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 66% (36115/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 67% (36662/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 68% (37209/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 69% (37757/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 70% (38304/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 71% (38851/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 72% (39398/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 73% (39945/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 74% (40493/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 75% (41040/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 76% (41587/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 77% (42134/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 78% (42681/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 79% (43229/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 80% (43776/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 81% (44323/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 82% (44870/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 83% (45417/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 84% (45964/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 85% (46512/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 86% (47059/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 87% (47606/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 88% (48153/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 89% (48700/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 90% (49248/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 91% (49795/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 92% (50342/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 93% (50889/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 94% (51436/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 95% (51984/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 96% (52531/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 97% (53078/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 98% (53625/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 99% (54172/54719), 156.78 MiB | 34.48 MiB/s remote: Total 54719 (delta 44), reused 28 (delta 28), pack-reused 54641 (from 3)
  7. 5.50 s Receiving objects: 100% (54719/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 100% (54719/54719), 168.30 MiB | 34.63 MiB/s, done.
  8. 5.71 s Resolving deltas: 0% (0/26547) Resolving deltas: 1% (266/26547) Resolving deltas: 2% (531/26547) Resolving deltas: 3% (797/26547) Resolving deltas: 4% (1063/26547) Resolving deltas: 5% (1328/26547) Resolving deltas: 6% (1593/26547) Resolving deltas: 7% (1860/26547) Resolving deltas: 8% (2125/26547) Resolving deltas: 9% (2390/26547) Resolving deltas: 10% (2655/26547) Resolving deltas: 11% (2921/26547) Resolving deltas: 12% (3186/26547) Resolving deltas: 13% (3452/26547) Resolving deltas: 14% (3717/26547) Resolving deltas: 15% (3983/26547) Resolving deltas: 16% (4248/26547) Resolving deltas: 17% (4513/26547) Resolving deltas: 18% (4779/26547) Resolving deltas: 19% (5044/26547) Resolving deltas: 20% (5311/26547) Resolving deltas: 21% (5575/26547) Resolving deltas: 22% (5841/26547) Resolving deltas: 23% (6106/26547) Resolving deltas: 24% (6373/26547) Resolving deltas: 25% (6637/26547) Resolving deltas: 26% (6903/26547) Resolving deltas: 27% (7168/26547) Resolving deltas: 28% (7434/26547) Resolving deltas: 29% (7699/26547) Resolving deltas: 30% (7965/26547) Resolving deltas: 31% (8230/26547) Resolving deltas: 32% (8496/26547) Resolving deltas: 33% (8761/26547) Resolving deltas: 34% (9026/26547) Resolving deltas: 35% (9292/26547) Resolving deltas: 36% (9557/26547) Resolving deltas: 37% (9825/26547) Resolving deltas: 38% (10090/26547) Resolving deltas: 39% (10354/26547) Resolving deltas: 40% (10622/26547) Resolving deltas: 41% (10885/26547) Resolving deltas: 42% (11150/26547) Resolving deltas: 43% (11416/26547) Resolving deltas: 44% (11681/26547) Resolving deltas: 45% (11947/26547) Resolving deltas: 46% (12212/26547) Resolving deltas: 47% (12478/26547) Resolving deltas: 48% (12744/26547) Resolving deltas: 49% (13009/26547) Resolving deltas: 50% (13274/26547) Resolving deltas: 51% (13539/26547) Resolving deltas: 52% (13805/26547) Resolving deltas: 53% (14070/26547) Resolving deltas: 54% (14337/26547) Resolving deltas: 55% (14601/26547) Resolving deltas: 56% (14868/26547) Resolving deltas: 57% (15132/26547) Resolving deltas: 58% (15398/26547) Resolving deltas: 59% (15665/26547) Resolving deltas: 60% (15929/26547) Resolving deltas: 61% (16194/26547) Resolving deltas: 62% (16460/26547) Resolving deltas: 63% (16725/26547) Resolving deltas: 64% (16991/26547) Resolving deltas: 65% (17256/26547) Resolving deltas: 66% (17522/26547) Resolving deltas: 67% (17787/26547) Resolving deltas: 68% (18052/26547) Resolving deltas: 69% (18318/26547) Resolving deltas: 70% (18583/26547) Resolving deltas: 71% (18849/26547) Resolving deltas: 72% (19114/26547) Resolving deltas: 73% (19380/26547) Resolving deltas: 74% (19645/26547) Resolving deltas: 75% (19911/26547) Resolving deltas: 76% (20176/26547) Resolving deltas: 77% (20442/26547) Resolving deltas: 78% (20707/26547) Resolving deltas: 79% (20973/26547) Resolving deltas: 80% (21238/26547) Resolving deltas: 81% (21505/26547) Resolving deltas: 82% (21769/26547) Resolving deltas: 83% (22036/26547) Resolving deltas: 84% (22300/26547) Resolving deltas: 85% (22565/26547) Resolving deltas: 86% (22831/26547) Resolving deltas: 87% (23096/26547) Resolving deltas: 88% (23362/26547) Resolving deltas: 89% (23627/26547) Resolving deltas: 90% (23893/26547) Resolving deltas: 91% (24158/26547) Resolving deltas: 92% (24424/26547) Resolving deltas: 93% (24690/26547) Resolving deltas: 94% (24955/26547) Resolving deltas: 95% (25221/26547) Resolving deltas: 96% (25486/26547) Resolving deltas: 97% (25751/26547) Resolving deltas: 98% (26019/26547) Resolving deltas: 99% (26282/26547) Resolving deltas: 100% (26547/26547) Resolving deltas: 100% (26547/26547), done.
  9. 6.09 s Fetching commit e923b9b20ed1f97069be27765c93a4cf20c28489 for refs/pull/1/merge ...
  10. 6.10 s $ /nix/store/bcnisk3ydfgv26v2gw3zlky24g00yww2-git-2.54.0/bin/git fetch origin e923b9b20ed1f97069be27765c93a4cf20c28489
  11. 6.57 s remote: Enumerating objects: 1, done.
  12. 6.58 s remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 1 (from 1)
  13. 6.59 s Unpacking objects:
  14. 6.59 s 100% (1/1) 100% (1/1), 875 bytes | 875.00 KiB/s, done.
  15. 6.60 s From https://github.com/NorfairKing/dspy
  16. 6.60 s * branch e923b9b20ed1f97069be27765c93a4cf20c28489 -> FETCH_HEAD
  17. 6.62 s $ /nix/store/9ypz3flqsrl5xl495mm8h645gadjsxi1-coreutils-9.11/bin/timeout --kill-after=15s 7200s /nix/store/bcnisk3ydfgv26v2gw3zlky24g00yww2-git-2.54.0/bin/git -c advice.detachedHead=false checkout e923b9b20ed1f97069be27765c93a4cf20c28489
  18. 6.68 s HEAD is now at e923b9b2 Merge 453ecf257a343742170b3e1cef20271508c3e6a4 into 498760149b230f402c56bece2aa45df6e1ba946b
  19. 6.69 s $ /nix/store/9ypz3flqsrl5xl495mm8h645gadjsxi1-coreutils-9.11/bin/timeout --kill-after=15s 7200s /nix/store/23zk8sgzamrbmj1fday15szf7f2279s8-nix-2.34.7/bin/nix --extra-experimental-features nix-command --extra-experimental-features flakes --log-format internal-json run .#packages.x86_64-linux.impure-test --print-build-logs
  20. 8.15 s
  21. 9.91 s ============================= test session starts ==============================
  22. 9.91 s platform linux -- Python 3.13.13, pytest-8.4.2, pluggy-1.6.0 -- /nix/store/rn0r480kscbk8159qi3qjflil71kqglq-dspy-test-env/bin/python3.13
  23. 9.91 s rootdir: /tmp/nix-ci-worker/stable-pwd/repo
  24. 9.91 s configfile: pyproject.toml
  25. 9.91 s plugins: anyio-4.13.0, asyncio-1.3.0, typeguard-4.5.1, mock-3.14.1
  26. 9.91 s asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
  27. 14.16 s collecting ... collecting 6 items collecting 288 items collecting 604 items collecting 1156 items collected 1322 items / 1 skipped
  28. 14.19 s
  29. 15.63 s tests/adapters/test_adapter_utils.py::test_parse_value_str_annotation PASSED [ 0%]
  30. 17.25 s tests/adapters/test_adapter_utils.py::test_parse_value_pydantic_types PASSED [ 0%]
  31. 18.75 s tests/adapters/test_adapter_utils.py::test_parse_value_basic_types PASSED [ 0%]
  32. 20.16 s tests/adapters/test_adapter_utils.py::test_parse_value_literal PASSED [ 0%]
  33. 22.00 s tests/adapters/test_adapter_utils.py::test_parse_value_union PASSED [ 0%]
  34. 23.44 s tests/adapters/test_adapter_utils.py::test_parse_value_json_repair PASSED [ 0%]
  35. 24.92 s tests/adapters/test_audio.py::test_normalize_audio_format[wav-wav] PASSED [ 0%]
  36. 26.35 s tests/adapters/test_audio.py::test_normalize_audio_format[mp3-mp3] PASSED [ 0%]
  37. 27.80 s tests/adapters/test_audio.py::test_normalize_audio_format[x-wav-wav] PASSED [ 0%]
  38. 29.28 s tests/adapters/test_audio.py::test_normalize_audio_format[x-mp3-mp3] PASSED [ 0%]
  39. 30.67 s tests/adapters/test_audio.py::test_normalize_audio_format[x-flac-flac] PASSED [ 0%]
  40. 32.02 s tests/adapters/test_audio.py::test_normalize_audio_format[my-x-format-my-x-format] PASSED [ 0%]
  41. 33.41 s tests/adapters/test_audio.py::test_normalize_audio_format[x-my-format-my-format] PASSED [ 0%]
  42. 34.93 s tests/adapters/test_audio.py::test_normalize_audio_format[-] PASSED [ 1%]
  43. 37.12 s tests/adapters/test_audio.py::test_normalize_audio_format[x--] PASSED [ 1%]
  44. 39.33 s tests/adapters/test_baml_adapter.py::test_baml_adapter_format_exact_messages_for_simple_signature_with_demo PASSED [ 1%]
  45. 41.56 s tests/adapters/test_baml_adapter.py::test_baml_adapter_format_exact_messages_with_nested_output PASSED [ 1%]
  46. 43.65 s tests/adapters/test_baml_adapter.py::test_baml_adapter_basic_schema_generation PASSED [ 1%]
  47. 45.84 s tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_optional_fields PASSED [ 1%]
  48. 48.21 s tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_primitive_types PASSED [ 1%]
  49. 50.32 s tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_lists_with_bracket_notation PASSED [ 1%]
  50. 52.50 s tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_complex_nested_models PASSED [ 1%]
  51. 54.61 s tests/adapters/test_baml_adapter.py::test_baml_adapter_raise_error_on_circular_references PASSED [ 1%]
  52. 56.82 s tests/adapters/test_baml_adapter.py::test_baml_adapter_formats_pydantic_inputs_as_clean_json PASSED [ 1%]
  53. 58.90 s tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_mixed_input_types PASSED [ 1%]
  54. 61.09 s tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_schema_generation_errors_gracefully PASSED [ 1%]
  55. 63.29 s tests/adapters/test_baml_adapter.py::test_baml_adapter_raises_on_missing_fields PASSED [ 2%]
  56. 65.30 s tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_type_casting_errors PASSED [ 2%]
  57. 67.48 s tests/adapters/test_baml_adapter.py::test_baml_adapter_with_images PASSED [ 2%]
  58. 69.72 s tests/adapters/test_baml_adapter.py::test_baml_adapter_with_tools PASSED [ 2%]
  59. 72.00 s tests/adapters/test_baml_adapter.py::test_baml_adapter_with_code PASSED [ 2%]
  60. 74.21 s tests/adapters/test_baml_adapter.py::test_baml_adapter_with_conversation_history PASSED [ 2%]
  61. 76.30 s tests/adapters/test_baml_adapter.py::test_baml_vs_json_adapter_token_efficiency PASSED [ 2%]
  62. 78.38 s tests/adapters/test_baml_adapter.py::test_baml_vs_json_adapter_functional_compatibility PASSED [ 2%]
  63. 80.58 s tests/adapters/test_baml_adapter.py::test_baml_adapter_async_functionality PASSED [ 2%]
  64. 82.69 s tests/adapters/test_baml_adapter.py::test_baml_adapter_with_field_aliases PASSED [ 2%]
  65. 84.90 s tests/adapters/test_baml_adapter.py::test_baml_adapter_field_alias_without_description PASSED [ 2%]
  66. 87.05 s tests/adapters/test_baml_adapter.py::test_baml_adapter_multiple_pydantic_input_fields PASSED [ 2%]
  67. 89.09 s tests/adapters/test_base_type.py::test_basic_extract_custom_type_from_annotation PASSED [ 2%]
  68. 91.28 s tests/adapters/test_base_type.py::test_extract_custom_type_from_annotation_with_nested_type PASSED [ 3%]
  69. 93.60 s tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-two-Literal['one', 'two', 'three"']-Literal['four', 'five', 'six"']] PASSED [ 3%]
  70. 95.76 s tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-she's here-Literal["she's here", 'okay', 'test']-Literal['done', "maybe'soon", 'later']] PASSED [ 3%]
  71. 97.97 s tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-another-Literal['both"and\\'', 'another']-Literal['yet"another\\'', 'plain']] PASSED [ 3%]
  72. 100.06 s tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-foo-Literal['foo', 'bar']-Literal['baz', 'qux']] PASSED [ 3%]
  73. 102.24 s tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-bar-Literal[1, 'bar']-Literal[True, 3, 'foo']] PASSED [ 3%]
  74. 104.42 s tests/adapters/test_chat_adapter.py::test_chat_adapter_sync_call PASSED [ 3%]
  75. 106.56 s tests/adapters/test_chat_adapter.py::test_chat_adapter_async_call PASSED [ 3%]
  76. 108.73 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_for_simple_signature PASSED [ 3%]
  77. 110.84 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_demo_and_typed_outputs PASSED [ 3%]
  78. 113.02 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_nested_pydantic_models PASSED [ 3%]
  79. 115.25 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_incomplete_demo PASSED [ 3%]
  80. 117.44 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_history PASSED [ 3%]
  81. 119.53 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_list_value_for_string_input PASSED [ 4%]
  82. 121.72 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_literal_output PASSED [ 4%]
  83. 123.75 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_multimodal_custom_type_inputs PASSED [ 4%]
  84. 125.85 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_history_demo_pydantic_tools_and_image PASSED [ 4%]
  85. 127.98 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_base_custom_type_input PASSED [ 4%]
  86. 129.92 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_citations_output_demo PASSED [ 4%]
  87. 131.79 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_and_lm_kwargs_with_native_citations PASSED [ 4%]
  88. 133.73 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_preserves_passthrough_lm_kwargs PASSED [ 4%]
  89. 135.57 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_and_lm_kwargs_with_native_reasoning PASSED [ 4%]
  90. 137.48 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_calling_still_enables_native_reasoning PASSED [ 4%]
  91. 139.39 s tests/adapters/test_chat_adapter.py::test_chat_adapter_nonnative_strips_native_tool_kwargs PASSED [ 4%]
  92. 141.24 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_reasoning_and_code_outputs PASSED [ 4%]
  93. 143.05 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_and_lm_kwargs_with_native_tool_calling PASSED [ 4%]
  94. 144.90 s tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_can_request_parallel_tool_calls[adapter0] PASSED [ 4%]
  95. 146.77 s tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_can_request_parallel_tool_calls[adapter1] PASSED [ 5%]
  96. 148.53 s tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_can_request_parallel_tool_calls[adapter2] PASSED [ 5%]
  97. 150.15 s tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_respects_lm_kwargs_parallel_tool_call_override PASSED [ 5%]
  98. 151.85 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_replay PASSED [ 5%]
  99. 153.57 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_replays_parallel_tool_results PASSED [ 5%]
  100. 155.29 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_empty_user_message PASSED [ 5%]
  101. 157.04 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_unmatched_tool_calls[call_1-None] PASSED [ 5%]
  102. 158.83 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_unmatched_tool_calls[call_1-tool_call_results1] PASSED [ 5%]
  103. 160.59 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_unmatched_tool_calls[None-tool_call_results2] PASSED [ 5%]
  104. 162.37 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_non_native_tool_history PASSED [ 5%]
  105. 164.04 s tests/adapters/test_chat_adapter.py::test_non_native_tool_history_remains_text_based[adapter0] PASSED [ 5%]
  106. 165.74 s tests/adapters/test_chat_adapter.py::test_non_native_tool_history_remains_text_based[adapter1] PASSED [ 5%]
  107. 167.47 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_accepts_custom_history_formatter_returning_messages_only PASSED [ 5%]
  108. 169.24 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_tool_input PASSED [ 6%]
  109. 170.92 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_kitchen_sink PASSED [ 6%]
  110. 172.73 s tests/adapters/test_chat_adapter.py::test_chat_adapter_with_pydantic_models PASSED [ 6%]
  111. 174.48 s tests/adapters/test_chat_adapter.py::test_chat_adapter_signature_information PASSED [ 6%]
  112. 176.17 s tests/adapters/test_chat_adapter.py::test_chat_adapter_exception_raised_on_failure PASSED [ 6%]
  113. 177.81 s tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image PASSED [ 6%]
  114. 179.54 s tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image_with_few_shot_examples PASSED [ 6%]
  115. 181.27 s tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image_with_nested_images PASSED [ 6%]
  116. 183.05 s tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image_with_few_shot_examples_with_nested_images PASSED [ 6%]
  117. 184.69 s tests/adapters/test_chat_adapter.py::test_chat_adapter_with_tool PASSED [ 6%]
  118. 186.47 s tests/adapters/test_chat_adapter.py::test_chat_adapter_with_code PASSED [ 6%]
  119. 188.13 s tests/adapters/test_chat_adapter.py::test_code_output_field_omits_json_schema_in_prompt PASSED [ 6%]
  120. 189.90 s tests/adapters/test_chat_adapter.py::test_citations_output_field_keeps_json_schema_in_prompt PASSED [ 6%]
  121. 191.53 s tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_conversation_history PASSED [ 7%]
  122. 193.12 s tests/adapters/test_chat_adapter.py::test_chat_adapter_fallback_to_json_adapter_on_exception PASSED [ 7%]
  123. 194.78 s tests/adapters/test_chat_adapter.py::test_chat_adapter_fallback_preserves_native_function_calling_flag PASSED [ 7%]
  124. 196.44 s tests/adapters/test_chat_adapter.py::test_chat_adapter_respects_use_json_adapter_fallback_flag PASSED [ 7%]
  125. 198.14 s tests/adapters/test_chat_adapter.py::test_chat_adapter_fallback_to_json_adapter_on_exception_async PASSED [ 7%]
  126. 199.89 s tests/adapters/test_chat_adapter.py::test_chat_adapter_async_fallback_preserves_native_function_calling_flag PASSED [ 7%]
  127. 201.55 s tests/adapters/test_chat_adapter.py::test_chat_adapter_toolcalls_native_function_calling PASSED [ 7%]
  128. 203.19 s tests/adapters/test_chat_adapter.py::test_chat_adapter_toolcalls_vague_match PASSED [ 7%]
  129. 204.94 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_reasoning PASSED [ 7%]
  130. 206.73 s tests/adapters/test_chat_adapter.py::test_chat_adapter_parses_float_with_underscores PASSED [ 7%]
  131. 208.35 s tests/adapters/test_chat_adapter.py::test_format_system_message PASSED [ 7%]
  132. 210.00 s tests/adapters/test_chat_adapter.py::test_null_content_raises_adapter_parse_error PASSED [ 7%]
  133. 211.64 s tests/adapters/test_chat_adapter.py::test_empty_string_content_raises_adapter_parse_error PASSED [ 7%]
  134. 213.31 s tests/adapters/test_chat_adapter.py::test_tool_call_with_null_content_does_not_raise PASSED [ 8%]
  135. 215.13 s tests/adapters/test_chat_adapter.py::test_tool_call_with_unstructured_content_does_not_raise PASSED [ 8%]
  136. 216.85 s tests/adapters/test_chat_adapter.py::test_tool_call_with_structured_content_preserves_other_outputs PASSED [ 8%]
  137. 218.59 s tests/adapters/test_chat_adapter.py::test_provider_tool_calls_preserve_id_and_repair_arguments PASSED [ 8%]
  138. 220.28 s tests/adapters/test_citation.py::test_citation_validate_input PASSED [ 8%]
  139. 222.06 s tests/adapters/test_citation.py::test_citations_in_nested_type PASSED [ 8%]
  140. 223.82 s tests/adapters/test_citation.py::test_citation_with_all_fields PASSED [ 8%]
  141. 225.59 s tests/adapters/test_citation.py::test_citation_format PASSED [ 8%]
  142. 227.23 s tests/adapters/test_citation.py::test_citations_format PASSED [ 8%]
  143. 228.86 s tests/adapters/test_citation.py::test_citations_from_dict_list PASSED [ 8%]
  144. 230.49 s tests/adapters/test_citation.py::test_citations_postprocessing PASSED [ 8%]
  145. 232.13 s tests/adapters/test_citation.py::test_citation_extraction_from_lm_response PASSED [ 8%]
  146. 233.83 s tests/adapters/test_code.py::test_code_validate_input PASSED [ 8%]
  147. 235.56 s tests/adapters/test_code.py::test_code_in_nested_type PASSED [ 9%]
  148. 237.25 s tests/adapters/test_code.py::test_code_with_language PASSED [ 9%]
  149. 238.83 s tests/adapters/test_code.py::test_code_parses_from_dirty_code PASSED [ 9%]
  150. 240.24 s tests/adapters/test_document.py::test_document_validate_input PASSED [ 9%]
  151. 241.60 s tests/adapters/test_document.py::test_document_in_nested_type PASSED [ 9%]
  152. 242.97 s tests/adapters/test_document.py::test_document_with_all_fields PASSED [ 9%]
  153. 244.83 s tests/adapters/test_document.py::test_document_format PASSED [ 9%]
  154. 246.55 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_for_simple_signature PASSED [ 9%]
  155. 248.27 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_demo_and_typed_output PASSED [ 9%]
  156. 250.08 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_described_and_bool_outputs PASSED [ 9%]
  157. 251.76 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_history_demo_pydantic_tools_and_image PASSED [ 9%]
  158. 253.53 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_int_and_mapping_outputs PASSED [ 9%]
  159. 255.41 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_literal_and_enum_outputs PASSED [ 9%]
  160. 257.05 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_nested_pydantic_output PASSED [ 9%]
  161. 258.67 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_incomplete_demo PASSED [ 10%]
  162. 260.48 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_and_lm_kwargs_with_native_tool_calling PASSED [ 10%]
  163. 262.33 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_tool_calls_output_demo PASSED [ 10%]
  164. 264.03 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_non_native_tool_result_history_field PASSED [ 10%]
  165. 265.76 s tests/adapters/test_json_adapter.py::test_json_adapter_passes_structured_output_when_supported_by_model PASSED [ 10%]
  166. 267.30 s tests/adapters/test_json_adapter.py::test_json_adapter_not_using_structured_outputs_when_not_supported_by_model PASSED [ 10%]
  167. 268.62 s tests/adapters/test_json_adapter.py::test_json_adapter_with_structured_outputs_does_not_mutate_original_signature PASSED [ 10%]
  168. 269.90 s tests/adapters/test_json_adapter.py::test_json_adapter_sync_call PASSED [ 10%]
  169. 271.39 s tests/adapters/test_json_adapter.py::test_json_adapter_async_call PASSED [ 10%]
  170. 272.66 s tests/adapters/test_json_adapter.py::test_json_adapter_on_pydantic_model PASSED [ 10%]
  171. 273.92 s tests/adapters/test_json_adapter.py::test_json_adapter_parse_raise_error_on_mismatch_fields PASSED [ 10%]
  172. 275.48 s tests/adapters/test_json_adapter.py::test_json_adapter_formats_image PASSED [ 10%]
  173. 277.18 s tests/adapters/test_json_adapter.py::test_json_adapter_formats_image_with_few_shot_examples PASSED [ 10%]
  174. 278.87 s tests/adapters/test_json_adapter.py::test_json_adapter_formats_image_with_nested_images PASSED [ 11%]
  175. 280.59 s tests/adapters/test_json_adapter.py::test_json_adapter_formats_with_nested_documents PASSED [ 11%]
  176. 282.33 s tests/adapters/test_json_adapter.py::test_json_adapter_formats_image_with_few_shot_examples_with_nested_images PASSED [ 11%]
  177. 283.68 s tests/adapters/test_json_adapter.py::test_json_adapter_with_tool PASSED [ 11%]
  178. 284.95 s tests/adapters/test_json_adapter.py::test_json_adapter_with_code PASSED [ 11%]
  179. 286.26 s tests/adapters/test_json_adapter.py::test_json_adapter_formats_conversation_history PASSED [ 11%]
  180. 287.88 s tests/adapters/test_json_adapter.py::test_json_adapter_on_pydantic_model_async PASSED [ 11%]
  181. 289.56 s tests/adapters/test_json_adapter.py::test_json_adapter_does_not_fallback_to_json_mode_on_structured_output_lm_error PASSED [ 11%]
  182. 291.28 s tests/adapters/test_json_adapter.py::test_json_adapter_json_mode_no_structured_outputs PASSED [ 11%]
  183. 293.12 s tests/adapters/test_json_adapter.py::test_json_adapter_json_mode_no_structured_outputs_async PASSED [ 11%]
  184. 294.80 s tests/adapters/test_json_adapter.py::test_json_adapter_does_not_fallback_to_json_mode_on_structured_output_lm_error_async PASSED [ 11%]
  185. 296.54 s tests/adapters/test_json_adapter.py::test_error_message_on_json_adapter_failure PASSED [ 11%]
  186. 298.30 s tests/adapters/test_json_adapter.py::test_error_message_on_json_adapter_failure_async PASSED [ 11%]
  187. 300.04 s tests/adapters/test_json_adapter.py::test_json_adapter_toolcalls_native_function_calling PASSED [ 12%]
  188. 301.72 s tests/adapters/test_json_adapter.py::test_json_adapter_toolcalls_no_native_function_calling PASSED [ 12%]
  189. 303.45 s tests/adapters/test_json_adapter.py::test_json_adapter_native_reasoning PASSED [ 12%]
  190. 305.21 s tests/adapters/test_json_adapter.py::test_json_adapter_with_responses_api PASSED [ 12%]
  191. 307.01 s tests/adapters/test_json_adapter.py::test_format_system_message PASSED [ 12%]
  192. 308.75 s tests/adapters/test_reasoning.py::test_reasoning_basic_operations PASSED [ 12%]
  193. 310.47 s tests/adapters/test_reasoning.py::test_reasoning_concatenation PASSED [ 12%]
  194. 312.24 s tests/adapters/test_reasoning.py::test_reasoning_string_methods PASSED [ 12%]
  195. 313.99 s tests/adapters/test_reasoning.py::test_reasoning_with_chain_of_thought PASSED [ 12%]
  196. 315.73 s tests/adapters/test_reasoning.py::test_reasoning_error_message PASSED [ 12%]
  197. 317.53 s tests/adapters/test_tool.py::test_basic_initialization PASSED [ 12%]
  198. 319.40 s tests/adapters/test_tool.py::test_tool_from_function PASSED [ 12%]
  199. 321.12 s tests/adapters/test_tool.py::test_tool_from_class PASSED [ 12%]
  200. 322.89 s tests/adapters/test_tool.py::test_tool_from_function_with_pydantic PASSED [ 13%]
  201. 324.61 s tests/adapters/test_tool.py::test_tool_from_function_with_pydantic_nesting PASSED [ 13%]
  202. 326.41 s tests/adapters/test_tool.py::test_tool_callable PASSED [ 13%]
  203. 328.35 s tests/adapters/test_tool.py::test_tool_with_pydantic_callable PASSED [ 13%]
  204. 330.19 s tests/adapters/test_tool.py::test_invalid_function_call PASSED [ 13%]
  205. 331.86 s tests/adapters/test_tool.py::test_parameter_desc PASSED [ 13%]
  206. 333.64 s tests/adapters/test_tool.py::test_tool_with_default_args_without_type_hints PASSED [ 13%]
  207. 335.28 s tests/adapters/test_tool.py::test_tool_call_parses_args PASSED [ 13%]
  208. 336.59 s tests/adapters/test_tool.py::test_tool_call_parses_nested_list_of_pydantic_model PASSED [ 13%]
  209. 337.93 s tests/adapters/test_tool.py::test_tool_call_kwarg PASSED [ 13%]
  210. 339.55 s tests/adapters/test_tool.py::test_tool_str PASSED [ 13%]
  211. 341.40 s tests/adapters/test_tool.py::test_async_tool_from_function PASSED [ 13%]
  212. 342.90 s tests/adapters/test_tool.py::test_async_tool_with_pydantic PASSED [ 13%]
  213. 344.51 s tests/adapters/test_tool.py::test_async_tool_with_complex_pydantic PASSED [ 13%]
  214. 346.42 s tests/adapters/test_tool.py::test_async_tool_invalid_call PASSED [ 14%]
  215. 348.22 s tests/adapters/test_tool.py::test_async_tool_with_kwargs PASSED [ 14%]
  216. 350.22 s tests/adapters/test_tool.py::test_async_concurrent_calls PASSED [ 14%]
  217. 352.21 s tests/adapters/test_tool.py::test_async_tool_call_in_sync_mode PASSED [ 14%]
  218. 354.02 s tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data0-expected0] PASSED [ 14%]
  219. 356.04 s tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data1-expected1] PASSED [ 14%]
  220. 357.92 s tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data2-expected2] PASSED [ 14%]
  221. 359.84 s tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data3-expected3] PASSED [ 14%]
  222. 361.75 s tests/adapters/test_tool.py::test_tool_calls_format_from_dict_list PASSED [ 14%]
  223. 363.41 s tests/adapters/test_tool.py::test_tool_calls_preserves_ids_from_dict_list_and_format_omits_ids PASSED [ 14%]
  224. 365.21 s tests/adapters/test_tool.py::test_tool_calls_json_schema_omits_internal_id_field PASSED [ 14%]
  225. 366.96 s tests/adapters/test_tool.py::test_tool_calls_can_carry_results_without_formatting_them_for_lm PASSED [ 14%]
  226. 368.73 s tests/adapters/test_tool.py::test_tool_call_results_from_tool_calls_and_values PASSED [ 14%]
  227. 370.54 s tests/adapters/test_tool.py::test_tool_call_results_history_serialization_round_trip PASSED [ 15%]
  228. 372.30 s tests/adapters/test_tool.py::test_tool_call_results_can_round_trip_as_native_tool_result_message PASSED [ 15%]
  229. 374.06 s tests/adapters/test_tool.py::test_toolcalls_vague_match PASSED [ 15%]
  230. 375.77 s tests/adapters/test_tool.py::test_tool_convert_input_schema_to_tool_args_no_input_params PASSED [ 15%]
  231. 377.45 s tests/adapters/test_tool.py::test_tool_convert_input_schema_to_tool_args_lang_chain PASSED [ 15%]
  232. 378.92 s tests/adapters/test_tool.py::test_tool_call_execute PASSED [ 15%]
  233. 380.90 s tests/adapters/test_tool.py::test_tool_call_execute_with_local_functions PASSED [ 15%]
  234. 382.36 s tests/adapters/test_two_step_adapter.py::test_two_step_adapter_format_exact_messages_for_simple_signature_with_demo PASSED [ 15%]
  235. 384.27 s tests/adapters/test_two_step_adapter.py::test_two_step_adapter_format_exact_messages_with_typed_outputs PASSED [ 15%]
  236. 386.02 s tests/adapters/test_two_step_adapter.py::test_two_step_adapter_call PASSED [ 15%]
  237. 387.69 s tests/adapters/test_two_step_adapter.py::test_two_step_adapter_async_call PASSED [ 15%]
  238. 389.44 s tests/adapters/test_two_step_adapter.py::test_two_step_adapter_parse PASSED [ 15%]
  239. 391.26 s tests/adapters/test_two_step_adapter.py::test_two_step_adapter_parse_errors PASSED [ 15%]
  240. 392.99 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_and_parse_basic PASSED [ 16%]
  241. 394.78 s tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_multiple_fields PASSED [ 16%]
  242. 396.52 s tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_raises_on_missing_field PASSED [ 16%]
  243. 398.19 s tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_casts_types PASSED [ 16%]
  244. 399.92 s tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_raises_on_type_error PASSED [ 16%]
  245. 401.35 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_and_parse_nested_model PASSED [ 16%]
  246. 402.77 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_and_parse_list_of_models PASSED [ 16%]
  247. 404.16 s tests/adapters/test_xml_adapter.py::test_xml_adapter_with_tool_like_output PASSED [ 16%]
  248. 405.81 s tests/adapters/test_xml_adapter.py::test_xml_adapter_formats_nested_images PASSED [ 16%]
  249. 407.50 s tests/adapters/test_xml_adapter.py::test_xml_adapter_with_code PASSED [ 16%]
  250. 409.34 s tests/adapters/test_xml_adapter.py::test_xml_adapter_full_prompt PASSED [ 16%]
  251. 411.02 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_for_simple_signature PASSED [ 16%]
  252. 412.70 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_non_native_tool_result_history_field PASSED [ 16%]
  253. 414.47 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_for_two_input_signature PASSED [ 17%]
  254. 416.26 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_demo_and_typed_output PASSED [ 17%]
  255. 417.97 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_history_demo_pydantic_tools_and_image PASSED [ 17%]
  256. 419.75 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_nested_pydantic_output PASSED [ 17%]
  257. 421.53 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_incomplete_demo PASSED [ 17%]
  258. 423.19 s tests/adapters/test_xml_adapter.py::test_format_system_message PASSED [ 17%]
  259. 424.80 s tests/callback/test_callback.py::test_callback_injection[args0-kwargs0] PASSED [ 17%]
  260. 426.57 s tests/callback/test_callback.py::test_callback_injection[args1-kwargs1] PASSED [ 17%]
  261. 428.04 s tests/callback/test_callback.py::test_callback_injection[args2-kwargs2] PASSED [ 17%]
  262. 429.65 s tests/callback/test_callback.py::test_callback_injection[args3-kwargs3] PASSED [ 17%]
  263. 431.27 s tests/callback/test_callback.py::test_callback_injection_local PASSED [ 17%]
  264. 432.77 s tests/callback/test_callback.py::test_callback_error_handling PASSED [ 17%]
  265. 434.32 s tests/callback/test_callback.py::test_multiple_callbacks PASSED [ 17%]
  266. 435.72 s tests/callback/test_callback.py::test_callback_complex_module PASSED [ 18%]
  267. 437.26 s tests/callback/test_callback.py::test_callback_async_module PASSED [ 18%]
  268. 438.70 s tests/callback/test_callback.py::test_tool_calls PASSED [ 18%]
  269. 440.10 s tests/callback/test_callback.py::test_active_id PASSED [ 18%]
  270. 442.89 s tests/clients/test_cache.py::test_initialization PASSED [ 18%]
  271. 444.30 s tests/clients/test_cache.py::test_invalid_cache_initialization PASSED [ 18%]
  272. 447.28 s tests/clients/test_cache.py::test_cache_key_generation PASSED [ 18%]
  273. 449.99 s tests/clients/test_cache.py::test_put_and_get PASSED [ 18%]
  274. 452.89 s tests/clients/test_cache.py::test_cache_miss PASSED [ 18%]
  275. 455.63 s tests/clients/test_cache.py::test_unserializable_key PASSED [ 18%]
  276. 458.57 s tests/clients/test_cache.py::test_reset_memory_cache PASSED [ 18%]
  277. 461.76 s tests/clients/test_cache.py::test_save_and_load_memory_cache PASSED [ 18%]
  278. 464.73 s tests/clients/test_cache.py::test_request_cache_decorator PASSED [ 18%]
  279. 467.71 s tests/clients/test_cache.py::test_request_cache_decorator_with_ignored_args_for_cache_key PASSED [ 18%]
  280. 470.48 s tests/clients/test_cache.py::test_request_cache_decorator_async PASSED [ 19%]
  281. 473.54 s tests/clients/test_cache.py::test_cache_consistency_with_lm_call_modifies_the_request PASSED [ 19%]
  282. 475.12 s tests/clients/test_cache.py::test_cache_fallback_on_restricted_environment PASSED [ 19%]
  283. 476.85 s tests/clients/test_cache.py::test_cache_init_with_disk_disabled_and_none_dir PASSED [ 19%]
  284. 479.71 s tests/clients/test_cache.py::test_model_response_roundtrip_in_restricted_mode PASSED [ 19%]
  285. 482.93 s tests/clients/test_cache.py::test_registered_dataclass_roundtrip_in_restricted_mode PASSED [ 19%]
  286. 486.12 s tests/clients/test_cache.py::test_configure_cache_registers_safe_types PASSED [ 19%]
  287. 488.12 s tests/clients/test_cache.py::test_corrupt_disk_entries_return_none PASSED [ 19%]
  288. 490.17 s tests/clients/test_cache.py::test_restricted_and_unrestricted_share_wire_format PASSED [ 19%]
  289. 492.11 s tests/clients/test_cache.py::test_unlisted_type_rejected_on_read PASSED [ 19%]
  290. 492.98 s tests/clients/test_cache.py::test_safe_types_rejects_non_types PASSED [ 19%]
  291. 492.98 s tests/clients/test_databricks.py::test_create_directory_in_databricks_unity_catalog SKIPPED [ 19%]
  292. 492.98 s tests/clients/test_databricks.py::test_create_finetuning_job SKIPPED [ 19%]
  293. 492.98 s tests/clients/test_databricks.py::test_deploy_finetuned_model SKIPPED [ 20%]
  294. 494.05 s tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[dict] PASSED [ 20%]
  295. 495.83 s tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[list] PASSED [ 20%]
  296. 497.72 s tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[str] PASSED [ 20%]
  297. 499.53 s tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[int] PASSED [ 20%]
  298. 500.78 s tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[tuple] PASSED [ 20%]
  299. 502.05 s tests/clients/test_disk_serialization.py::test_pydantic_roundtrip PASSED [ 20%]
  300. 503.16 s tests/clients/test_disk_serialization.py::test_dataclass_roundtrip PASSED [ 20%]
  301. 504.28 s tests/clients/test_disk_serialization.py::test_ndarray_roundtrip PASSED [ 20%]
  302. 505.39 s tests/clients/test_disk_serialization.py::test_litellm_response_roundtrip PASSED [ 20%]
  303. 506.99 s tests/clients/test_disk_serialization.py::test_text_completion_response_roundtrip PASSED [ 20%]
  304. 508.66 s tests/clients/test_disk_serialization.py::test_embedding_response_roundtrip PASSED [ 20%]
  305. 510.34 s tests/clients/test_disk_serialization.py::test_responses_api_roundtrip PASSED [ 20%]
  306. 512.23 s tests/clients/test_disk_serialization.py::test_tool_call_response_roundtrip PASSED [ 21%]
  307. 517.24 s tests/clients/test_disk_serialization.py::test_cached_tool_call_normalizes_after_fresh_process_roundtrip
  308. 518.35 s PASSED [ 21%]
  309. 523.36 s tests/clients/test_disk_serialization.py::test_cached_responses_function_call_normalizes_after_fresh_process_roundtrip
  310. 524.36 s PASSED [ 21%]
  311. 526.36 s tests/clients/test_disk_serialization.py::test_unlisted_type_blocked_on_read PASSED [ 21%]
  312. 528.41 s tests/clients/test_disk_serialization.py::test_allowlists_are_isolated PASSED [ 21%]
  313. 529.79 s tests/clients/test_disk_serialization.py::test_numpy_ctypeslib_blocked PASSED [ 21%]
  314. 531.13 s tests/clients/test_disk_serialization.py::test_restricted_load_rejects_unknown_type PASSED [ 21%]
  315. 532.64 s tests/clients/test_disk_serialization.py::test_corrupt_pickle_raises_deserialization_error PASSED [ 21%]
  316. 534.23 s tests/clients/test_disk_serialization.py::test_nested_class_safe_types PASSED [ 21%]
  317. 535.57 s tests/clients/test_disk_serialization.py::test_restricted_load_allows_builtin_types PASSED [ 21%]
  318. 537.01 s tests/clients/test_disk_serialization.py::test_all_cached_lm_types_roundtrip PASSED [ 21%]
  319. 538.76 s tests/clients/test_embedding.py::test_litellm_embedding PASSED [ 21%]
  320. 540.37 s tests/clients/test_embedding.py::test_callable_embedding PASSED [ 21%]
  321. 543.82 s tests/clients/test_embedding.py::test_callable_numpy_embedding_persists_to_disk PASSED [ 22%]
  322. 545.02 s tests/clients/test_embedding.py::test_invalid_model_type PASSED [ 22%]
  323. 546.51 s tests/clients/test_embedding.py::test_async_embedding PASSED [ 22%]
  324. 548.11 s tests/clients/test_embedding.py::test_call_caching_false_overrides_instance_true PASSED [ 22%]
  325. 549.88 s tests/clients/test_embedding.py::test_call_caching_true_overrides_instance_false PASSED [ 22%]
  326. 551.55 s tests/clients/test_embedding.py::test_acall_caching_false_overrides_instance_true PASSED [ 22%]
  327. 553.23 s tests/clients/test_embedding.py::test_acall_caching_true_overrides_instance_false PASSED [ 22%]
  328. 554.68 s tests/clients/test_inspect_global_history.py::test_inspect_history_basic PASSED [ 22%]
  329. 556.26 s tests/clients/test_inspect_global_history.py::test_inspect_history_renders_message_tool_calls PASSED [ 22%]
  330. 557.63 s tests/clients/test_inspect_global_history.py::test_inspect_history_renders_output_tool_calls_without_text PASSED [ 22%]
  331. 559.35 s tests/clients/test_inspect_global_history.py::test_inspect_history_with_n PASSED [ 22%]
  332. 560.72 s tests/clients/test_inspect_global_history.py::test_inspect_empty_history PASSED [ 22%]
  333. 562.24 s tests/clients/test_inspect_global_history.py::test_inspect_history_n_larger_than_history PASSED [ 22%]
  334. 563.62 s tests/clients/test_lazy_litellm_import.py::test_import_dspy_does_not_import_litellm PASSED [ 22%]
  335. 564.96 s tests/clients/test_lazy_litellm_import.py::test_lm_litellm_use_raises_helpful_error_without_litellm PASSED [ 23%]
  336. 566.48 s tests/clients/test_lazy_litellm_import.py::test_embedder_litellm_use_raises_helpful_error_without_litellm PASSED [ 23%]
  337. 567.98 s tests/clients/test_lazy_litellm_import.py::test_concurrent_lm_first_use_materializes_litellm_once PASSED [ 23%]
  338. 572.00 s tests/clients/test_lm.py::test_chat_lms_can_be_queried PASSED [ 23%]
  339. 577.02 s tests/clients/test_lm.py::test_dspy_cache
  340. 577.38 s PASSED [ 23%]
  341. 578.70 s tests/clients/test_lm.py::test_disabled_cache_skips_cache_key PASSED [ 23%]
  342. 581.53 s tests/clients/test_lm.py::test_rollout_id_bypasses_cache PASSED [ 23%]
  343. 582.92 s tests/clients/test_lm.py::test_zero_temperature_rollout_warns_once PASSED [ 23%]
  344. 584.48 s tests/clients/test_lm.py::test_rollout_id_with_default_temperature_does_not_warn PASSED [ 23%]
  345. 588.89 s tests/clients/test_lm.py::test_text_lms_can_be_queried PASSED [ 23%]
  346. 592.81 s tests/clients/test_lm.py::test_lm_calls_support_callables PASSED [ 23%]
  347. 596.75 s tests/clients/test_lm.py::test_lm_calls_support_pydantic_models PASSED [ 23%]
  348. 598.08 s tests/clients/test_lm.py::test_lm_wraps_litellm_errors_with_metadata PASSED [ 23%]
  349. 599.42 s tests/clients/test_lm.py::test_lm_wraps_litellm_context_window_error PASSED [ 24%]
  350. 600.83 s tests/clients/test_lm.py::test_lm_wraps_unknown_boundary_error_as_unexpected_error PASSED [ 24%]
  351. 602.36 s tests/clients/test_lm.py::test_lm_preserves_existing_lm_error_without_self_cause PASSED [ 24%]
  352. 603.74 s tests/clients/test_lm.py::test_lm_preserves_existing_lm_error_without_self_cause_async PASSED [ 24%]
  353. 605.14 s tests/clients/test_lm.py::test_retry_number_set_correctly PASSED [ 24%]
  354. 609.69 s tests/clients/test_lm.py::test_retry_made_on_system_errors PASSED [ 24%]
  355. 611.01 s tests/clients/test_lm.py::test_reasoning_model_token_parameter PASSED [ 24%]
  356. 612.47 s tests/clients/test_lm.py::test_reasoning_model_requirements[openai/o1] PASSED [ 24%]
  357. 613.95 s tests/clients/test_lm.py::test_reasoning_model_requirements[openai/gpt-5-nano] PASSED [ 24%]
  358. 615.28 s tests/clients/test_lm.py::test_reasoning_model_requirements[openai/gpt-5-mini] PASSED [ 24%]
  359. 616.78 s tests/clients/test_lm.py::test_gpt_5_chat_not_reasoning_model PASSED [ 24%]
  360. 618.14 s tests/clients/test_lm.py::test_base_lm_init_uses_lm_defaults_and_isolates_callback_list PASSED [ 24%]
  361. 619.43 s tests/clients/test_lm.py::test_base_lm_forward_contract_defaults_to_legacy PASSED [ 24%]
  362. 620.48 s tests/clients/test_lm.py::test_base_lm_forward_contract_accepts_explicit_values PASSED [ 25%]
  363. 621.43 s tests/clients/test_lm.py::test_base_lm_forward_contract_rejects_unknown_values PASSED [ 25%]
  364. 622.56 s tests/clients/test_lm.py::test_base_lm_validates_typed_lm_response PASSED [ 25%]
  365. 623.92 s tests/clients/test_lm.py::test_base_lm_warns_when_inherited_legacy_forward_returns_lm_response PASSED [ 25%]
  366. 625.30 s tests/clients/test_lm.py::test_base_lm_errors_when_explicit_legacy_forward_returns_lm_response PASSED [ 25%]
  367. 626.82 s tests/clients/test_lm.py::test_base_lm_inherited_legacy_forward_returning_lm_response_errors_on_direct_call PASSED [ 25%]
  368. 628.23 s tests/clients/test_lm.py::test_base_lm_default_call_keeps_legacy_outputs PASSED [ 25%]
  369. 629.63 s tests/clients/test_lm.py::test_base_lm_experimental_call_returns_lm_response_through_legacy_bridge PASSED [ 25%]
  370. 631.01 s tests/clients/test_lm.py::test_base_lm_explicit_lm_request_returns_lm_response_without_experimental PASSED [ 25%]
  371. 632.54 s tests/clients/test_lm.py::test_base_lm_legacy_bridge_records_typed_history_and_usage_once PASSED [ 25%]
  372. 633.94 s tests/clients/test_lm.py::test_base_lm_typed_forward_contract_uses_lm_request PASSED [ 25%]
  373. 635.32 s tests/clients/test_lm.py::test_base_lm_typed_forward_contract_rejects_non_lm_response_at_call_time PASSED [ 25%]
  374. 636.78 s tests/clients/test_lm.py::test_base_lm_request_call_rejects_mixed_inputs PASSED [ 25%]
  375. 638.10 s tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_system_user_and_assistant_turns[current_lm] PASSED [ 26%]
  376. 639.41 s tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_system_user_and_assistant_turns[typed_lm] PASSED [ 26%]
  377. 640.73 s tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_tool_call_transcripts[current_lm] PASSED [ 26%]
  378. 642.20 s tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_tool_call_transcripts[typed_lm] PASSED [ 26%]
  379. 643.54 s tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_can_reuse_lm_response_as_assistant_turn[current_lm] PASSED [ 26%]
  380. 644.90 s tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_can_reuse_lm_response_as_assistant_turn[typed_lm] PASSED [ 26%]
  381. 646.51 s tests/clients/test_lm.py::test_base_lm_async_explicit_lm_request_returns_lm_response PASSED [ 26%]
  382. 647.93 s tests/clients/test_lm.py::test_base_lm_tracks_usage_for_custom_subclasses PASSED [ 26%]
  383. 649.33 s tests/clients/test_lm.py::test_base_lm_copy_is_shallow_runtime_copy_with_isolated_dspy_state PASSED [ 26%]
  384. 650.67 s tests/clients/test_lm.py::test_dump_state PASSED [ 26%]
  385. 652.12 s tests/clients/test_lm.py::test_reasoning_model_dump_state_uses_constructor_max_tokens PASSED [ 26%]
  386. 653.45 s tests/clients/test_lm.py::test_dump_state_preserves_enabled_developer_role PASSED [ 26%]
  387. 654.79 s tests/clients/test_lm.py::test_dump_state_ignores_internal_class_marker_kwarg PASSED [ 26%]
  388. 656.29 s tests/clients/test_lm.py::test_load_state PASSED [ 27%]
  389. 657.65 s tests/clients/test_lm.py::test_reasoning_model_load_state_round_trips_canonical_state PASSED [ 27%]
  390. 659.00 s tests/clients/test_lm.py::test_reasoning_model_load_state_accepts_max_completion_tokens_alias PASSED [ 27%]
  391. 660.41 s tests/clients/test_lm.py::test_lm_load_state_forwards_allow_custom_lm_class PASSED [ 27%]
  392. 664.98 s tests/clients/test_lm.py::test_exponential_backoff_retry PASSED [ 27%]
  393. 666.53 s tests/clients/test_lm.py::test_logprobs_included_when_requested PASSED [ 27%]
  394. 667.85 s tests/clients/test_lm.py::test_async_lm_call PASSED [ 27%]
  395. 670.54 s tests/clients/test_lm.py::test_async_lm_call_with_cache PASSED [ 27%]
  396. 672.02 s tests/clients/test_lm.py::test_lm_history_size_limit PASSED [ 27%]
  397. 673.43 s tests/clients/test_lm.py::test_disable_history PASSED [ 27%]
  398. 674.77 s tests/clients/test_lm.py::test_responses_api PASSED [ 27%]
  399. 676.28 s tests/clients/test_lm.py::test_lm_replaces_system_with_developer_role PASSED [ 27%]
  400. 680.14 s tests/clients/test_lm.py::test_responses_api_tool_calls PASSED [ 27%]
  401. 681.65 s tests/clients/test_lm.py::test_reasoning_effort_responses_api PASSED [ 27%]
  402. 683.04 s tests/clients/test_lm.py::test_call_reasoning_model_with_chat_api PASSED [ 28%]
  403. 684.36 s tests/clients/test_lm.py::test_api_key_not_saved_in_json PASSED [ 28%]
  404. 685.68 s tests/clients/test_lm.py::test_responses_api_converts_images_correctly PASSED [ 28%]
  405. 687.17 s tests/clients/test_lm.py::test_responses_api_converts_files_correctly PASSED [ 28%]
  406. 688.52 s tests/clients/test_lm.py::test_responses_api_preserves_multi_message_structure PASSED [ 28%]
  407. 689.87 s tests/clients/test_lm.py::test_responses_api_with_image_input PASSED [ 28%]
  408. 691.36 s tests/clients/test_lm.py::test_responses_api_with_pydantic_model_input PASSED [ 28%]
  409. 692.55 s tests/clients/test_lm.py::test_responses_api_with_none_usage PASSED [ 28%]
  410. 693.95 s tests/clients/test_lm.py::test_responses_api_with_none_usage_async PASSED [ 28%]
  411. 695.32 s tests/clients/test_lm.py::test_streaming_passes_headers_correctly PASSED [ 28%]
  412. 695.32 s tests/clients/test_lm_direct_live.py::test_live_openai_chat_direct_system_user_assistant_multiturn SKIPPED [ 28%]
  413. 695.32 s tests/clients/test_lm_direct_live.py::test_live_openai_chat_direct_tool_call_transcript SKIPPED [ 28%]
  414. 695.33 s tests/clients/test_lm_direct_live.py::test_live_openai_chat_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 28%]
  415. 695.33 s tests/clients/test_lm_direct_live.py::test_live_openai_responses_direct_system_user_assistant_multiturn SKIPPED [ 29%]
  416. 695.33 s tests/clients/test_lm_direct_live.py::test_live_openai_responses_direct_tool_call_transcript SKIPPED [ 29%]
  417. 695.33 s tests/clients/test_lm_direct_live.py::test_live_openai_responses_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 29%]
  418. 695.33 s tests/clients/test_lm_direct_live.py::test_live_anthropic_chat_direct_system_user_assistant_multiturn SKIPPED [ 29%]
  419. 695.33 s tests/clients/test_lm_direct_live.py::test_live_anthropic_chat_direct_tool_call_transcript SKIPPED [ 29%]
  420. 695.33 s tests/clients/test_lm_direct_live.py::test_live_anthropic_chat_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 29%]
  421. 695.33 s tests/clients/test_lm_direct_live.py::test_live_gemini_chat_direct_system_user_assistant_multiturn SKIPPED [ 29%]
  422. 695.33 s tests/clients/test_lm_direct_live.py::test_live_gemini_chat_direct_tool_call_transcript SKIPPED [ 29%]
  423. 695.33 s tests/clients/test_lm_direct_live.py::test_live_gemini_chat_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 29%]
  424. 696.89 s tests/clients/test_lm_local.py::test_command_with_spaces_in_path PASSED [ 29%]
  425. 698.29 s tests/clients/test_lm_local.py::test_command_construction_prevents_injection PASSED [ 29%]
  426. 699.68 s tests/clients/test_lm_local.py::test_command_is_list_not_string PASSED [ 29%]
  427. 701.06 s tests/core/test_types.py::test_message_content_and_tool_calls_normalize_for_dspy_history_surface PASSED [ 29%]
  428. 702.53 s tests/core/test_types.py::test_tool_result_content_none_normalizes_to_empty_parts PASSED [ 30%]
  429. 703.84 s tests/core/test_types.py::test_audio_content_accepts_url_and_history_preserves_url PASSED [ 30%]
  430. 705.17 s tests/core/test_types.py::test_audio_content_defaults_null_format_to_wav PASSED [ 30%]
  431. 706.70 s tests/core/test_types.py::test_image_content_requires_mapping_with_url PASSED [ 30%]
  432. 708.04 s tests/core/test_types.py::test_video_data_round_trips_through_history_messages PASSED [ 30%]
  433. 709.39 s tests/core/test_types.py::test_document_source_url_stays_url_and_round_trips_through_history_messages PASSED [ 30%]
  434. 710.76 s tests/core/test_types.py::test_config_extensions_flatten_when_converted_to_legacy_kwargs PASSED [ 30%]
  435. 712.23 s tests/core/test_types.py::test_lm_kwargs_aliases_normalize_for_existing_dspy_lm_callers PASSED [ 30%]
  436. 713.59 s tests/core/test_types.py::test_nested_config_aliases_remain_supported_for_existing_interfaces PASSED [ 30%]
  437. 715.04 s tests/core/test_types.py::test_usage_normalizes_existing_user_visible_token_aliases PASSED [ 30%]
  438. 716.63 s tests/core/test_types.py::test_default_config_does_not_serialize_empty_stop_sequences PASSED [ 30%]
  439. 717.98 s tests/core/test_types.py::test_history_entry_exposes_existing_dict_style_properties PASSED [ 30%]
  440. 719.37 s tests/core/test_types.py::test_response_rejects_empty_outputs PASSED [ 30%]
  441. 720.73 s tests/core/test_types.py::test_output_to_value_preserves_redacted_thinking_part PASSED [ 31%]
  442. 722.23 s tests/core/test_types.py::test_stream_event_indices_must_be_non_negative PASSED [ 31%]
  443. 723.63 s tests/core/test_types.py::test_stream_builder_rejects_sparse_output_indices PASSED [ 31%]
  444. 724.98 s tests/core/test_types.py::test_stream_builder_rejects_sparse_part_indices PASSED [ 31%]
  445. 726.51 s tests/core/test_types.py::test_stream_builder_rejects_delta_type_changes PASSED [ 31%]
  446. 727.92 s tests/core/test_types.py::test_stream_builder_rejects_incomplete_tool_call_arguments PASSED [ 31%]
  447. 727.92 s tests/datasets/test_dataset.py::test_input_keys SKIPPED (need --extra
  448. 727.92 s option to run) [ 31%]
  449. 729.32 s tests/docs/test_mkdocs_links.py::test_nav_files_exist PASSED [ 31%]
  450. 730.72 s tests/evaluate/test_auto_evaluation.py::test_semantic_f1_returns_prediction_without_trace PASSED [ 31%]
  451. 732.25 s tests/evaluate/test_auto_evaluation.py::test_semantic_f1_returns_prediction_with_trace PASSED [ 31%]
  452. 733.60 s tests/evaluate/test_auto_evaluation.py::test_semantic_f1_score_value PASSED [ 31%]
  453. 734.46 s tests/evaluate/test_auto_evaluation.py::test_complete_and_grounded_returns_prediction_without_trace PASSED [ 31%]
  454. 735.32 s tests/evaluate/test_auto_evaluation.py::test_complete_and_grounded_returns_prediction_with_trace PASSED [ 31%]
  455. 736.44 s tests/evaluate/test_auto_evaluation.py::test_semantic_f1_prediction_can_be_compared PASSED [ 31%]
  456. 737.87 s tests/evaluate/test_evaluate.py::test_evaluate_initialization PASSED [ 32%]
  457. 739.26 s tests/evaluate/test_evaluate.py::test_evaluate_call PASSED [ 32%]
  458. 740.63 s tests/evaluate/test_evaluate.py::test_evaluate_single_thread_runs_in_main_thread PASSED [ 32%]
  459. 740.63 s tests/evaluate/test_evaluate.py::test_construct_result_df SKIPPED (need
  460. 740.63 s --extra option to run) [ 32%]
  461. 742.17 s tests/evaluate/test_evaluate.py::test_multithread_evaluate_call PASSED [ 32%]
  462. 744.72 s tests/evaluate/test_evaluate.py::test_multi_thread_evaluate_call_cancelled PASSED [ 32%]
  463. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_call_wrong_answer PASSED [ 32%]
  464. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example0] SKIPPED [ 32%]
  465. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example1] SKIPPED [ 32%]
  466. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example2] SKIPPED [ 32%]
  467. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example3] SKIPPED [ 32%]
  468. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example0] SKIPPED [ 32%]
  469. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example1] SKIPPED [ 32%]
  470. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example2] SKIPPED [ 33%]
  471. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example3] SKIPPED [ 33%]
  472. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example0] SKIPPED [ 33%]
  473. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example1] SKIPPED [ 33%]
  474. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example2] SKIPPED [ 33%]
  475. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example3] SKIPPED [ 33%]
  476. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example0] SKIPPED [ 33%]
  477. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example1] SKIPPED [ 33%]
  478. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example2] SKIPPED [ 33%]
  479. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example3] SKIPPED [ 33%]
  480. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example0] SKIPPED [ 33%]
  481. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example1] SKIPPED [ 33%]
  482. 746.28 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example2] SKIPPED [ 33%]
  483. 746.29 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example3] SKIPPED [ 34%]
  484. 746.29 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example0] SKIPPED [ 34%]
  485. 746.29 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example1] SKIPPED [ 34%]
  486. 746.29 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example2] SKIPPED [ 34%]
  487. 746.29 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example3] SKIPPED [ 34%]
  488. 747.75 s tests/evaluate/test_evaluate.py::test_evaluate_callback PASSED [ 34%]
  489. 749.16 s tests/evaluate/test_evaluate.py::test_evaluation_result_repr PASSED [ 34%]
  490. 750.52 s tests/evaluate/test_evaluate.py::test_evaluate_save_as_json_with_history PASSED [ 34%]
  491. 752.08 s tests/evaluate/test_evaluate.py::test_evaluate_save_as_csv_with_history PASSED [ 34%]
  492. 753.44 s tests/evaluate/test_metrics.py::test_answer_exact_match_string PASSED [ 34%]
  493. 754.84 s tests/evaluate/test_metrics.py::test_answer_exact_match_list PASSED [ 34%]
  494. 756.36 s tests/evaluate/test_metrics.py::test_answer_exact_match_no_match PASSED [ 34%]
  495. 757.58 s tests/metadata/test_metadata.py::test_metadata PASSED [ 34%]
  496. 758.95 s tests/predict/test_aggregation.py::test_majority_with_prediction PASSED [ 35%]
  497. 760.34 s tests/predict/test_aggregation.py::test_majority_with_completions PASSED [ 35%]
  498. 761.86 s tests/predict/test_aggregation.py::test_majority_with_list PASSED [ 35%]
  499. 763.19 s tests/predict/test_aggregation.py::test_majority_with_normalize PASSED [ 35%]
  500. 764.54 s tests/predict/test_aggregation.py::test_majority_with_field PASSED [ 35%]
  501. 765.88 s tests/predict/test_aggregation.py::test_majority_with_no_majority PASSED [ 35%]
  502. 767.38 s tests/predict/test_best_of_n.py::test_refine_forward_success_first_attempt PASSED [ 35%]
  503. 768.72 s tests/predict/test_best_of_n.py::test_refine_module_default_fail_count PASSED [ 35%]
  504. 770.09 s tests/predict/test_best_of_n.py::test_refine_module_custom_fail_count PASSED [ 35%]
  505. 771.55 s tests/predict/test_chain_of_thought.py::test_initialization_with_string_signature PASSED [ 35%]
  506. 772.91 s tests/predict/test_chain_of_thought.py::test_async_chain_of_thought PASSED [ 35%]
  507. 774.27 s tests/predict/test_chain_of_thought.py::test_chain_of_thought_with_native_reasoning PASSED [ 35%]
  508. 775.68 s tests/predict/test_chain_of_thought.py::test_chain_of_thought_with_manual_reasoning PASSED [ 35%]
  509. 775.69 s tests/predict/test_code_act.py::test_codeact_code_generation SKIPPED [ 36%]
  510. 775.69 s tests/predict/test_code_act.py::test_codeact_support_multiple_fields SKIPPED [ 36%]
  511. 775.69 s tests/predict/test_code_act.py::test_codeact_code_parse_failure SKIPPED [ 36%]
  512. 775.69 s tests/predict/test_code_act.py::test_codeact_code_execution_failure SKIPPED [ 36%]
  513. 775.69 s tests/predict/test_code_act.py::test_codeact_tool_validation SKIPPED [ 36%]
  514. 775.69 s tests/predict/test_knn.py::test_knn_initialization SKIPPED (need --extra
  515. 775.69 s option to run) [ 36%]
  516. 775.69 s tests/predict/test_knn.py::test_knn_query SKIPPED (need --extra option
  517. 775.69 s to run) [ 36%]
  518. 775.69 s tests/predict/test_knn.py::test_knn_query_specificity SKIPPED (need
  519. 775.69 s --extra option to run) [ 36%]
  520. 777.25 s tests/predict/test_multi_chain_comparison.py::test_basic_example PASSED [ 36%]
  521. 778.63 s tests/predict/test_parallel.py::test_parallel_module PASSED [ 36%]
  522. 780.01 s tests/predict/test_parallel.py::test_batch_module PASSED [ 36%]
  523. 781.53 s tests/predict/test_parallel.py::test_nested_parallel_module PASSED [ 36%]
  524. 782.88 s tests/predict/test_parallel.py::test_nested_batch_method PASSED [ 36%]
  525. 784.21 s tests/predict/test_parallel.py::test_batch_with_failed_examples PASSED [ 36%]
  526. 785.56 s tests/predict/test_parallel.py::test_parallel_timeout_and_straggler_limit_params PASSED [ 37%]
  527. 787.12 s tests/predict/test_parallel.py::test_batch_timeout_and_straggler_limit_params PASSED [ 37%]
  528. 788.50 s tests/predict/test_predict.py::test_initialization_with_string_signature PASSED [ 37%]
  529. 789.88 s tests/predict/test_predict.py::test_reset_method PASSED [ 37%]
  530. 791.43 s tests/predict/test_predict.py::test_lm_after_dump_and_load_state PASSED [ 37%]
  531. 792.76 s tests/predict/test_predict.py::test_base_lm_dump_state_ignores_internal_class_marker_kwarg PASSED [ 37%]
  532. 794.09 s tests/predict/test_predict.py::test_legacy_lm_state_without_class_marker_loads_as_lm PASSED [ 37%]
  533. 795.42 s tests/predict/test_predict.py::test_custom_lm_load_state_requires_trusted_opt_in PASSED [ 37%]
  534. 796.88 s tests/predict/test_predict.py::test_nested_custom_lm_class_path_loads_for_trusted_state PASSED [ 37%]
  535. 798.20 s tests/predict/test_predict.py::test_call_method PASSED [ 37%]
  536. 799.43 s tests/predict/test_predict.py::test_instructions_after_dump_and_load_state PASSED [ 37%]
  537. 800.40 s tests/predict/test_predict.py::test_demos_after_dump_and_load_state PASSED [ 37%]
  538. 801.85 s tests/predict/test_predict.py::test_typed_demos_after_dump_and_load_state PASSED [ 37%]
  539. 803.14 s tests/predict/test_predict.py::test_signature_fields_after_dump_and_load_state PASSED [ 38%]
  540. 804.48 s tests/predict/test_predict.py::test_lm_field_after_dump_and_load_state[model.json] PASSED [ 38%]
  541. 805.92 s tests/predict/test_predict.py::test_lm_field_after_dump_and_load_state[model.pkl] PASSED [ 38%]
  542. 807.31 s tests/predict/test_predict.py::test_load_ignores_serialized_endpoint_override_by_default[api_base] PASSED [ 38%]
  543. 808.67 s tests/predict/test_predict.py::test_load_ignores_serialized_endpoint_override_by_default[base_url] PASSED [ 38%]
  544. 810.01 s tests/predict/test_predict.py::test_load_allows_serialized_endpoint_override_with_opt_in[api_base] PASSED [ 38%]
  545. 811.38 s tests/predict/test_predict.py::test_load_allows_serialized_endpoint_override_with_opt_in[base_url] PASSED [ 38%]
  546. 812.75 s tests/predict/test_predict.py::test_load_state_ignores_serialized_endpoint_override_by_default[api_base] PASSED [ 38%]
  547. 814.04 s tests/predict/test_predict.py::test_load_state_ignores_serialized_endpoint_override_by_default[base_url] PASSED [ 38%]
  548. 815.29 s tests/predict/test_predict.py::test_load_state_allows_serialized_endpoint_override_with_opt_in[api_base] PASSED [ 38%]
  549. 816.81 s tests/predict/test_predict.py::test_load_state_allows_serialized_endpoint_override_with_opt_in[base_url] PASSED [ 38%]
  550. 818.16 s tests/predict/test_predict.py::test_load_state_ignores_serialized_model_list_endpoint_override_by_default PASSED [ 38%]
  551. 819.53 s tests/predict/test_predict.py::test_load_prevents_serialized_endpoint_override_reaching_litellm[api_base] PASSED [ 38%]
  552. 820.93 s tests/predict/test_predict.py::test_load_prevents_serialized_endpoint_override_reaching_litellm[base_url] PASSED [ 39%]
  553. 822.30 s tests/predict/test_predict.py::test_load_blocks_serialized_model_list_unless_opted_in PASSED [ 39%]
  554. 823.63 s tests/predict/test_predict.py::test_load_uses_env_api_key_without_honoring_serialized_endpoint_override PASSED [ 39%]
  555. 824.96 s tests/predict/test_predict.py::test_forward_method PASSED [ 39%]
  556. 826.41 s tests/predict/test_predict.py::test_forward_method2 PASSED [ 39%]
  557. 827.90 s tests/predict/test_predict.py::test_config_management PASSED [ 39%]
  558. 829.55 s tests/predict/test_predict.py::test_multi_output PASSED [ 39%]
  559. 831.32 s tests/predict/test_predict.py::test_multi_output2 PASSED [ 39%]
  560. 832.47 s tests/predict/test_predict.py::test_datetime_inputs_and_outputs PASSED [ 39%]
  561. 833.85 s tests/predict/test_predict.py::test_explicitly_valued_enum_inputs_and_outputs PASSED [ 39%]
  562. 835.24 s tests/predict/test_predict.py::test_enum_inputs_and_outputs_with_shared_names_and_values PASSED [ 39%]
  563. 836.80 s tests/predict/test_predict.py::test_auto_valued_enum_inputs_and_outputs PASSED [ 39%]
  564. 838.18 s tests/predict/test_predict.py::test_named_predictors PASSED [ 39%]
  565. 839.56 s tests/predict/test_predict.py::test_output_only PASSED [ 40%]
  566. 840.92 s tests/predict/test_predict.py::test_load_state_chaining PASSED [ 40%]
  567. 842.42 s tests/predict/test_predict.py::test_call_predict_with_chat_history[chat] PASSED [ 40%]
  568. 843.77 s tests/predict/test_predict.py::test_call_predict_with_chat_history[json] PASSED [ 40%]
  569. 845.13 s tests/predict/test_predict.py::test_lm_usage PASSED [ 40%]
  570. 847.17 s tests/predict/test_predict.py::test_lm_usage_with_parallel PASSED [ 40%]
  571. 849.55 s tests/predict/test_predict.py::test_lm_usage_with_async PASSED [ 40%]
  572. 850.91 s tests/predict/test_predict.py::test_positional_arguments PASSED [ 40%]
  573. 852.49 s tests/predict/test_predict.py::test_error_message_on_invalid_lm_setup PASSED [ 40%]
  574. 853.84 s tests/predict/test_predict.py::test_field_constraints[chat] PASSED [ 40%]
  575. 855.19 s tests/predict/test_predict.py::test_field_constraints[json] PASSED [ 40%]
  576. 856.66 s tests/predict/test_predict.py::test_async_predict PASSED [ 40%]
  577. 857.99 s tests/predict/test_predict.py::test_predicted_outputs_piped_from_predict_to_lm_call PASSED [ 40%]
  578. 859.30 s tests/predict/test_predict.py::test_dump_state_pydantic_non_primitive_types PASSED [ 40%]
  579. 860.81 s tests/predict/test_predict.py::test_trace_size_limit PASSED [ 41%]
  580. 862.28 s tests/predict/test_predict.py::test_disable_trace PASSED [ 41%]
  581. 863.67 s tests/predict/test_predict.py::test_per_module_history_size_limit PASSED [ 41%]
  582. 865.07 s tests/predict/test_predict.py::test_per_module_history_disabled PASSED [ 41%]
  583. 866.62 s tests/predict/test_predict.py::test_input_field_default_value PASSED [ 41%]
  584. 868.04 s tests/predict/test_predict.py::test_extra_fields_warning PASSED [ 41%]
  585. 869.29 s tests/predict/test_predict.py::test_missing_optional_input_field_no_warning PASSED [ 41%]
  586. 870.74 s tests/predict/test_predict.py::test_missing_required_input_field_still_warns PASSED [ 41%]
  587. 872.26 s tests/predict/test_predict.py::test_warning_images PASSED [ 41%]
  588. 873.62 s tests/predict/test_predict.py::test_type_mismatch_warning PASSED [ 41%]
  589. 874.98 s tests/predict/test_predict.py::test_correct_types_no_warning PASSED [ 41%]
  590. 876.51 s tests/predict/test_predict.py::test_list_type_validation PASSED [ 41%]
  591. 877.91 s tests/predict/test_predict.py::test_literal_type_validation PASSED [ 41%]
  592. 879.34 s tests/predict/test_predict.py::test_literal_union_type_validation PASSED [ 42%]
  593. 880.59 s tests/predict/test_predict.py::test_list_string PASSED [ 42%]
  594. 881.88 s tests/predict/test_predict.py::test_nested_list_type_validation PASSED [ 42%]
  595. 883.29 s tests/predict/test_predict.py::test_nested_dict_type_validation PASSED [ 42%]
  596. 884.70 s tests/predict/test_predict.py::test_nested_tuple_type_validation PASSED [ 42%]
  597. 886.28 s tests/predict/test_predict.py::test_literal_type_validation_string_signature PASSED [ 42%]
  598. 887.62 s tests/predict/test_predict.py::test_list_type_validation_string_signature PASSED [ 42%]
  599. 888.96 s tests/predict/test_predict.py::test_dict_type_validation_string_signature PASSED [ 42%]
  600. 890.30 s tests/predict/test_predict.py::test_tuple_type_validation_string_signature PASSED [ 42%]
  601. 891.78 s tests/predict/test_predict.py::test_union_type_validation_string_signature PASSED [ 42%]
  602. 892.99 s tests/predict/test_predict.py::test_basic_types_string_signature[False] PASSED [ 42%]
  603. 894.34 s tests/predict/test_predict.py::test_basic_types_string_signature[True] PASSED [ 42%]
  604. 895.67 s tests/predict/test_predict.py::test_untyped_string_signature PASSED [ 42%]
  605. 897.20 s tests/predict/test_predict.py::test_untyped_class_signature PASSED [ 43%]
  606. 898.58 s tests/predict/test_predict.py::test_string_to_list_signature PASSED [ 43%]
  607. 900.00 s tests/predict/test_predict.py::test_custom_signature_types[False] PASSED [ 43%]
  608. 901.61 s tests/predict/test_predict.py::test_custom_signature_types[True] PASSED [ 43%]
  609. 901.61 s tests/predict/test_program_of_thought.py::test_pot_code_generation SKIPPED [ 43%]
  610. 901.61 s tests/predict/test_program_of_thought.py::test_old_style_pot SKIPPED [ 43%]
  611. 901.61 s tests/predict/test_program_of_thought.py::test_pot_support_multiple_fields SKIPPED [ 43%]
  612. 901.61 s tests/predict/test_program_of_thought.py::test_pot_code_generation_with_one_error SKIPPED [ 43%]
  613. 901.61 s tests/predict/test_program_of_thought.py::test_pot_code_generation_persistent_errors SKIPPED [ 43%]
  614. 903.01 s tests/predict/test_program_of_thought.py::test_pot_code_parse_error PASSED [ 43%]
  615. 903.01 s tests/predict/test_react.py::test_tool_observation_preserves_custom_type SKIPPED [ 43%]
  616. 904.29 s tests/predict/test_react.py::test_tool_calling_with_pydantic_args PASSED [ 43%]
  617. 905.67 s tests/predict/test_react.py::test_react_with_tools_skips_native_response_issubclass_for_generic_alias PASSED [ 43%]
  618. 907.20 s tests/predict/test_react.py::test_tool_calling_without_typehint PASSED [ 44%]
  619. 908.56 s tests/predict/test_react.py::test_trajectory_truncation PASSED [ 44%]
  620. 910.04 s tests/predict/test_react.py::test_context_window_exceeded_after_retries PASSED [ 44%]
  621. 911.56 s tests/predict/test_react.py::test_error_retry PASSED [ 44%]
  622. 912.95 s tests/predict/test_react.py::test_async_tool_calling_with_pydantic_args PASSED [ 44%]
  623. 914.37 s tests/predict/test_react.py::test_async_error_retry PASSED [ 44%]
  624. 915.90 s tests/predict/test_react_v2.py::test_react_v2_submit_tool_returns_original_output_fields PASSED [ 44%]
  625. 917.38 s tests/predict/test_react_v2.py::test_react_v2_text_mock_lm_loop_records_inputs_once PASSED [ 44%]
  626. 918.75 s tests/predict/test_react_v2.py::test_react_v2_continuation_omits_missing_original_inputs PASSED [ 44%]
  627. 920.11 s tests/predict/test_react_v2.py::test_react_v2_text_mode_accepts_top_level_tool_arguments PASSED [ 44%]
  628. 921.58 s tests/predict/test_react_v2.py::test_react_v2_text_mode_accepts_wrapped_submit_arguments PASSED [ 44%]
  629. 923.03 s tests/predict/test_react_v2.py::test_react_v2_unknown_tool_observation_can_continue PASSED [ 44%]
  630. 924.52 s tests/predict/test_react_v2.py::test_react_v2_accepts_serialized_history_input PASSED [ 44%]
  631. 925.88 s tests/predict/test_react_v2.py::test_react_v2_forced_submit_on_empty_tool_calls PASSED [ 45%]
  632. 927.43 s tests/predict/test_react_v2.py::test_react_v2_native_tool_loop_replays_tool_result_with_provider_id PASSED [ 45%]
  633. 928.77 s tests/predict/test_react_v2.py::test_react_v2_native_parallel_tool_calls_are_requested_and_replayed PASSED [ 45%]
  634. 930.19 s tests/predict/test_refine.py::test_refine_forward_success_first_attempt PASSED [ 45%]
  635. 931.73 s tests/predict/test_refine.py::test_refine_module_default_fail_count PASSED [ 45%]
  636. 933.15 s tests/predict/test_refine.py::test_refine_module_custom_fail_count PASSED [ 45%]
  637. 934.50 s tests/predict/test_rlm.py::TestMockInterpreter::test_scripted_responses PASSED [ 45%]
  638. 935.83 s tests/predict/test_rlm.py::TestMockInterpreter::test_returns_final_output_result PASSED [ 45%]
  639. 937.20 s tests/predict/test_rlm.py::TestMockInterpreter::test_raises_exception_from_responses PASSED [ 45%]
  640. 938.57 s tests/predict/test_rlm.py::TestMockInterpreter::test_records_call_history PASSED [ 45%]
  641. 939.96 s tests/predict/test_rlm.py::TestRLMInitialization::test_basic_initialization PASSED [ 45%]
  642. 941.47 s tests/predict/test_rlm.py::TestRLMInitialization::test_custom_signature PASSED [ 45%]
  643. 942.86 s tests/predict/test_rlm.py::TestRLMInitialization::test_custom_tools PASSED [ 45%]
  644. 944.27 s tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_invalid_identifier[invalid-name] PASSED [ 45%]
  645. 945.67 s tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_invalid_identifier[123start] PASSED [ 46%]
  646. 947.12 s tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_reserved_names[llm_query] PASSED [ 46%]
  647. 948.41 s tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_reserved_names[SUBMIT] PASSED [ 46%]
  648. 949.73 s tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_reserved_names[print] PASSED [ 46%]
  649. 951.04 s tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_not_callable[not a function] PASSED [ 46%]
  650. 952.51 s tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_not_callable[123] PASSED [ 46%]
  651. 953.95 s tests/predict/test_rlm.py::TestRLMInitialization::test_tools_dict_rejected PASSED [ 46%]
  652. 955.30 s tests/predict/test_rlm.py::TestRLMInitialization::test_optional_parameters PASSED [ 46%]
  653. 956.88 s tests/predict/test_rlm.py::TestRLMInitialization::test_forward_validates_required_inputs PASSED [ 46%]
  654. 958.16 s tests/predict/test_rlm.py::TestRLMInitialization::test_batched_query_errors_have_clear_markers PASSED [ 46%]
  655. 959.51 s tests/predict/test_rlm.py::TestRLMInitialization::test_tools_call_counter_is_thread_safe PASSED [ 46%]
  656. 960.85 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```python\nprint(1)\n```-print(1)] PASSED [ 46%]
  657. 962.39 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```py\nx = 1\nprint(x)\n```-x = 1\nprint(x)] PASSED [ 46%]
  658. 963.88 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```\nprint('no lang')\n```-print('no lang')] PASSED [ 47%]
  659. 965.25 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[not fenced code-not fenced code] PASSED [ 47%]
  660. 966.76 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[I'll inspect first.\n```python\nprint('hello')\n```\nThen I will submit.-print('hello')] PASSED [ 47%]
  661. 968.15 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```python\nprint(1)\n```\nsome trailing text-print(1)] PASSED [ 47%]
  662. 969.53 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```python\nprint('oops')-print('oops')] PASSED [ 47%]
  663. 970.90 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```\n```python\nprint(1)\n```\n```-print(1)] PASSED [ 47%]
  664. 972.42 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```\n```\nprint(2)\n```\n```-print(2)] PASSED [ 47%]
  665. 973.82 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences_rejects_non_python_lang PASSED [ 47%]
  666. 975.21 s tests/predict/test_rlm.py::TestRLMFormatting::test_format_history PASSED [ 47%]
  667. 976.73 s tests/predict/test_rlm.py::TestRLMFormatting::test_format_history_empty PASSED [ 47%]
  668. 978.12 s tests/predict/test_rlm.py::TestRLMFormatting::test_action_signature_has_iteration_field PASSED [ 47%]
  669. 979.49 s tests/predict/test_rlm.py::TestRLMFormatting::test_format_output PASSED [ 47%]
  670. 980.81 s tests/predict/test_rlm.py::TestRLMFormatting::test_format_output_empty PASSED [ 47%]
  671. 982.32 s tests/predict/test_rlm.py::TestRLMFormatting::test_format_output_passthrough PASSED [ 48%]
  672. 983.75 s tests/predict/test_rlm.py::TestRLMFormatting::test_format_variable_info_string PASSED [ 48%]
  673. 985.08 s tests/predict/test_rlm.py::TestRLMFormatting::test_format_variable_info_dict PASSED [ 48%]
  674. 986.61 s tests/predict/test_rlm.py::TestRLMFormatting::test_build_variables_multiple PASSED [ 48%]
  675. 988.04 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_history_immutability PASSED [ 48%]
  676. 989.46 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_history_len_iter_bool PASSED [ 48%]
  677. 990.69 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_entry_format PASSED [ 48%]
  678. 992.26 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_entry_format_truncation PASSED [ 48%]
  679. 993.62 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_entry_format_no_truncation PASSED [ 48%]
  680. 994.96 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_history_threads_max_output_chars PASSED [ 48%]
  681. 996.48 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_from_value PASSED [ 48%]
  682. 997.83 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_truncation PASSED [ 48%]
  683. 999.17 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_with_field_info PASSED [ 48%]
  684. 1000.55 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_without_field_info PASSED [ 49%]
  685. 1002.08 s tests/predict/test_rlm.py::TestREPLTypes::test_build_variables_includes_field_metadata PASSED [ 49%]
  686. 1003.49 s tests/predict/test_rlm.py::TestRLMCallMethod::test_call_is_alias_for_forward PASSED [ 49%]
  687. 1004.89 s tests/predict/test_rlm.py::TestRLMMaxIterationsFallback::test_max_iters_triggers_extract PASSED [ 49%]
  688. 1006.45 s tests/predict/test_rlm.py::TestRLMToolExceptions::test_tool_exception_returns_error_in_output PASSED [ 49%]
  689. 1007.77 s tests/predict/test_rlm.py::TestRLMToolExceptions::test_runtime_error_history_uses_stripped_code PASSED [ 49%]
  690. 1009.11 s tests/predict/test_rlm.py::TestRLMToolExceptions::test_syntax_error_from_execute_is_recoverable PASSED [ 49%]
  691. 1010.44 s tests/predict/test_rlm.py::TestRLMToolExceptions::test_syntax_error_from_strip_code_fences_is_recoverable PASSED [ 49%]
  692. 1011.90 s tests/predict/test_rlm.py::TestRLMDynamicSignature::test_action_signature_structure PASSED [ 49%]
  693. 1013.23 s tests/predict/test_rlm.py::TestRLMDynamicSignature::test_extract_signature_structure PASSED [ 49%]
  694. 1013.26 s tests/predict/test_rlm.py::TestPythonInterpreter::test_start_prewarms_sandbox SKIPPED [ 49%]
  695. 1013.26 s tests/predict/test_rlm.py::TestPythonInterpreter::test_start_is_idempotent SKIPPED [ 49%]
  696. 1013.26 s tests/predict/test_rlm.py::TestPythonInterpreter::test_basic_execution SKIPPED [ 49%]
  697. 1013.26 s tests/predict/test_rlm.py::TestPythonInterpreter::test_variable_injection SKIPPED [ 50%]
  698. 1013.26 s tests/predict/test_rlm.py::TestPythonInterpreter::test_variable_injection_with_none_values SKIPPED [ 50%]
  699. 1013.26 s tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_call_kwargs SKIPPED [ 50%]
  700. 1013.26 s tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_call_positional SKIPPED [ 50%]
  701. 1013.26 s tests/predict/test_rlm.py::TestPythonInterpreter::test_multiple_tools SKIPPED [ 50%]
  702. 1013.26 s tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_returns_list SKIPPED [ 50%]
  703. 1013.26 s tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_returns_dict SKIPPED [ 50%]
  704. 1013.26 s tests/predict/test_rlm.py::TestPythonInterpreter::test_state_persists SKIPPED [ 50%]
  705. 1013.26 s tests/predict/test_rlm.py::TestPythonInterpreter::test_syntax_error SKIPPED [ 50%]
  706. 1013.26 s tests/predict/test_rlm.py::TestPythonInterpreter::test_runtime_error SKIPPED [ 50%]
  707. 1013.26 s tests/predict/test_rlm.py::TestSandboxSecurity::test_no_network_access SKIPPED [ 50%]
  708. 1013.26 s tests/predict/test_rlm.py::TestSandboxSecurity::test_imports_work SKIPPED [ 50%]
  709. 1014.73 s tests/predict/test_rlm.py::TestRLMAsyncMock::test_aforward_basic PASSED [ 50%]
  710. 1016.23 s tests/predict/test_rlm.py::TestRLMAsyncMock::test_aforward_int_output_mock PASSED [ 50%]
  711. 1017.67 s tests/predict/test_rlm.py::TestRLMAsyncMock::test_aforward_multi_iteration_mock PASSED [ 51%]
  712. 1019.11 s tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[count-int-42-SUBMIT(42)-42] PASSED [ 51%]
  713. 1020.55 s tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[score-float-3.14-SUBMIT(3.14)-3.14] PASSED [ 51%]
  714. 1022.06 s tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[valid-bool-True-SUBMIT(True)-True] PASSED [ 51%]
  715. 1023.45 s tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[numbers-list[int]-final_value3-SUBMIT([1, 2, 3])-expected3] PASSED [ 51%]
  716. 1024.80 s tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[answer-Literal['yes', 'no']-yes-SUBMIT("yes")-yes] PASSED [ 51%]
  717. 1026.33 s tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_error_retries PASSED [ 51%]
  718. 1026.34 s tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[count-int-SUBMIT(42)-42-int] SKIPPED [ 51%]
  719. 1026.34 s tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[score-float-SUBMIT(3.14)-3.14-float] SKIPPED [ 51%]
  720. 1026.34 s tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[valid-bool-SUBMIT(True)-True-bool] SKIPPED [ 51%]
  721. 1026.34 s tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[numbers-list[int]-SUBMIT([1, 2, 3])-expected3-list] SKIPPED [ 51%]
  722. 1026.34 s tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[data-dict[str, str]-SUBMIT({"key": "value"})-expected4-dict] SKIPPED [ 51%]
  723. 1026.34 s tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[answer-Literal['yes', 'no']-SUBMIT("yes")-yes-str] SKIPPED [ 51%]
  724. 1026.34 s tests/predict/test_rlm.py::TestRLMTypeCoercion::test_submit_extracts_typed_value SKIPPED [ 52%]
  725. 1026.34 s tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_final_kwargs SKIPPED [ 52%]
  726. 1026.34 s tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_final_positional SKIPPED [ 52%]
  727. 1026.34 s tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_three_fields SKIPPED [ 52%]
  728. 1026.34 s tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_final_missing_field_errors SKIPPED [ 52%]
  729. 1026.34 s tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_submit_vars SKIPPED [ 52%]
  730. 1026.34 s tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_type_coercion SKIPPED [ 52%]
  731. 1026.34 s tests/predict/test_rlm.py::TestRLMWithDummyLM::test_simple_computation_e2e SKIPPED [ 52%]
  732. 1026.34 s tests/predict/test_rlm.py::TestRLMWithDummyLM::test_multi_turn_computation_e2e SKIPPED [ 52%]
  733. 1026.34 s tests/predict/test_rlm.py::TestRLMWithDummyLM::test_with_input_variables_e2e SKIPPED [ 52%]
  734. 1026.34 s tests/predict/test_rlm.py::TestRLMWithDummyLM::test_with_tool_e2e SKIPPED [ 52%]
  735. 1026.34 s tests/predict/test_rlm.py::TestRLMWithDummyLM::test_aforward_simple_computation_e2e SKIPPED [ 52%]
  736. 1026.34 s tests/predict/test_rlm.py::TestRLMWithDummyLM::test_aforward_multi_turn_e2e SKIPPED [ 52%]
  737. 1026.34 s tests/predict/test_rlm.py::TestRLMWithDummyLM::test_aforward_with_input_variables_e2e SKIPPED [ 53%]
  738. 1026.34 s tests/predict/test_rlm.py::TestRLMIntegration::test_simple_computation SKIPPED [ 53%]
  739. 1026.34 s tests/predict/test_rlm.py::TestRLMIntegration::test_with_llm_query SKIPPED [ 53%]
  740. 1027.71 s tests/predict/test_rlm.py::TestBuildVariablesWithSerializable::test_serializable_uses_build_repl_variable PASSED [ 53%]
  741. 1029.10 s tests/predict/test_rlm.py::TestBuildVariablesWithSerializable::test_regular_values_unchanged PASSED [ 53%]
  742. 1030.55 s tests/predict/test_rlm.py::TestPrepareSerializableVars::test_separates_serializable_from_regular PASSED [ 53%]
  743. 1032.09 s tests/predict/test_rlm.py::TestPrepareSerializableVars::test_no_serializable_returns_all PASSED [ 53%]
  744. 1033.49 s tests/predict/test_rlm.py::TestPrepareSerializableVars::test_binary_payload_uses_base64_transport PASSED [ 53%]
  745. 1034.90 s tests/predict/test_rlm.py::TestPrepareSerializableVars::test_large_payload_not_inlined_in_code PASSED [ 53%]
  746. 1036.44 s tests/predict/test_rlm.py::TestPrepareSerializableVars::test_forward_with_serializable PASSED [ 53%]
  747. 1036.48 s tests/predict/test_rlm.py::TestLargeSerializableRoundTrip::test_large_payload_round_trips_through_real_sandbox SKIPPED [ 53%]
  748. 1037.76 s tests/primitives/test_base_module.py::test_deepcopy_basic PASSED [ 53%]
  749. 1039.07 s tests/primitives/test_base_module.py::test_deepcopy_with_uncopyable_modules PASSED [ 53%]
  750. 1040.40 s tests/primitives/test_base_module.py::test_deepcopy_with_nested_modules PASSED [ 54%]
  751. 1041.86 s tests/primitives/test_base_module.py::test_save_and_load_with_json PASSED [ 54%]
  752. 1041.90 s tests/primitives/test_base_module.py::test_save_and_load_with_pkl SKIPPED [ 54%]
  753. 1043.21 s tests/primitives/test_base_module.py::test_save_with_extra_modules PASSED [ 54%]
  754. 1044.56 s tests/primitives/test_base_module.py::test_load_with_version_mismatch PASSED [ 54%]
  755. 1044.56 s tests/primitives/test_base_module.py::test_single_module_call_with_usage_tracker SKIPPED [ 54%]
  756. 1044.56 s tests/primitives/test_base_module.py::test_multi_module_call_with_usage_tracker SKIPPED [ 54%]
  757. 1044.56 s tests/primitives/test_base_module.py::test_usage_tracker_in_parallel SKIPPED [ 54%]
  758. 1046.10 s tests/primitives/test_base_module.py::test_usage_tracker_async_parallel PASSED [ 54%]
  759. 1047.47 s tests/primitives/test_base_module.py::test_usage_tracker_no_side_effect PASSED [ 54%]
  760. 1048.91 s tests/primitives/test_base_module.py::test_module_history PASSED [ 54%]
  761. 1050.13 s tests/primitives/test_base_module.py::test_module_history_with_concurrency PASSED [ 54%]
  762. 1051.70 s tests/primitives/test_base_module.py::test_module_history_async PASSED [ 54%]
  763. 1053.05 s tests/primitives/test_base_module.py::test_forward_direct_call_warning PASSED [ 54%]
  764. 1054.41 s tests/primitives/test_base_module.py::test_forward_through_call_no_warning PASSED [ 55%]
  765. 1055.75 s tests/primitives/test_example.py::test_example_initialization PASSED [ 55%]
  766. 1057.24 s tests/primitives/test_example.py::test_example_initialization_from_base PASSED [ 55%]
  767. 1058.59 s tests/primitives/test_example.py::test_example_initialization_from_dict PASSED [ 55%]
  768. 1059.93 s tests/primitives/test_example.py::test_example_set_get_item PASSED [ 55%]
  769. 1061.42 s tests/primitives/test_example.py::test_example_attribute_access PASSED [ 55%]
  770. 1062.80 s tests/primitives/test_example.py::test_example_deletion PASSED [ 55%]
  771. 1064.17 s tests/primitives/test_example.py::test_example_len PASSED [ 55%]
  772. 1065.56 s tests/primitives/test_example.py::test_example_repr_str_img PASSED [ 55%]
  773. 1067.14 s tests/primitives/test_example.py::test_example_repr_str PASSED [ 55%]
  774. 1068.46 s tests/primitives/test_example.py::test_example_eq PASSED [ 55%]
  775. 1069.78 s tests/primitives/test_example.py::test_example_hash PASSED [ 55%]
  776. 1071.13 s tests/primitives/test_example.py::test_example_keys_values_items PASSED [ 55%]
  777. 1072.56 s tests/primitives/test_example.py::test_example_get PASSED [ 56%]
  778. 1073.96 s tests/primitives/test_example.py::test_example_with_inputs PASSED [ 56%]
  779. 1075.36 s tests/primitives/test_example.py::test_example_inputs_labels PASSED [ 56%]
  780. 1076.86 s tests/primitives/test_example.py::test_example_copy_without PASSED [ 56%]
  781. 1078.19 s tests/primitives/test_example.py::test_example_to_dict PASSED [ 56%]
  782. 1079.54 s tests/primitives/test_example.py::test_example_to_dict_with_history PASSED [ 56%]
  783. 1080.89 s tests/primitives/test_module.py::test_module_initialization PASSED [ 56%]
  784. 1082.40 s tests/primitives/test_module.py::test_named_predictors PASSED [ 56%]
  785. 1083.75 s tests/primitives/test_module.py::test_predictors PASSED [ 56%]
  786. 1085.06 s tests/primitives/test_module.py::test_forward PASSED [ 56%]
  787. 1086.57 s tests/primitives/test_module.py::test_nested_named_predictors PASSED [ 56%]
  788. 1087.92 s tests/primitives/test_module.py::test_empty_module PASSED [ 56%]
  789. 1089.27 s tests/primitives/test_module.py::test_single_level PASSED [ 56%]
  790. 1090.61 s tests/primitives/test_module.py::test_multiple_levels PASSED [ 57%]
  791. 1092.08 s tests/primitives/test_module.py::test_multiple_sub_modules PASSED [ 57%]
  792. 1093.41 s tests/primitives/test_module.py::test_non_base_module_attributes PASSED [ 57%]
  793. 1094.81 s tests/primitives/test_module.py::test_complex_module_traversal PASSED [ 57%]
  794. 1096.35 s tests/primitives/test_module.py::test_complex_module_traversal_with_same_module PASSED [ 57%]
  795. 1097.75 s tests/primitives/test_module.py::test_complex_module_set_attribute_by_name PASSED [ 57%]
  796. 1099.15 s tests/primitives/test_module.py::test_named_parameters_duplicate_references PASSED [ 57%]
  797. 1100.60 s tests/primitives/test_module.py::test_load_dspy_program_cross_version PASSED [ 57%]
  798. 1102.07 s tests/primitives/test_module.py::test_load_state_is_transactional PASSED [ 57%]
  799. 1102.07 s tests/primitives/test_python_interpreter.py::test_execute_simple_code SKIPPED [ 57%]
  800. 1102.07 s tests/primitives/test_python_interpreter.py::test_import SKIPPED (need
  801. 1102.07 s --deno option to run) [ 57%]
  802. 1102.07 s tests/primitives/test_python_interpreter.py::test_user_variable_definitions SKIPPED [ 57%]
  803. 1102.07 s tests/primitives/test_python_interpreter.py::test_rejects_python_keywords_as_variable_names SKIPPED [ 57%]
  804. 1102.07 s tests/primitives/test_python_interpreter.py::test_failure_syntax_error SKIPPED [ 58%]
  805. 1102.07 s tests/primitives/test_python_interpreter.py::test_failure_zero_division SKIPPED [ 58%]
  806. 1102.07 s tests/primitives/test_python_interpreter.py::test_exception_args SKIPPED [ 58%]
  807. 1102.07 s tests/primitives/test_python_interpreter.py::test_submit_with_list SKIPPED [ 58%]
  808. 1102.07 s tests/primitives/test_python_interpreter.py::test_enable_env_vars_flag SKIPPED [ 58%]
  809. 1102.08 s tests/primitives/test_python_interpreter.py::test_read_file_access_control SKIPPED [ 58%]
  810. 1102.08 s tests/primitives/test_python_interpreter.py::test_enable_write_flag SKIPPED [ 58%]
  811. 1102.08 s tests/primitives/test_python_interpreter.py::test_enable_net_flag SKIPPED [ 58%]
  812. 1102.08 s tests/primitives/test_python_interpreter.py::test_interpreter_security_filesystem_access SKIPPED [ 58%]
  813. 1102.08 s tests/primitives/test_python_interpreter.py::test_tools_dict_is_copied SKIPPED [ 58%]
  814. 1102.08 s tests/primitives/test_python_interpreter.py::test_serialize_tuple SKIPPED [ 58%]
  815. 1102.08 s tests/primitives/test_python_interpreter.py::test_serialize_set SKIPPED [ 58%]
  816. 1102.08 s tests/primitives/test_python_interpreter.py::test_serialize_set_mixed_types SKIPPED [ 58%]
  817. 1102.08 s tests/primitives/test_python_interpreter.py::test_deno_command_dict_raises_type_error SKIPPED [ 59%]
  818. 1102.08 s tests/primitives/test_python_interpreter.py::test_tool_with_typed_signature SKIPPED [ 59%]
  819. 1102.08 s tests/primitives/test_python_interpreter.py::test_tool_positional_args SKIPPED [ 59%]
  820. 1102.08 s tests/primitives/test_python_interpreter.py::test_tool_keyword_args SKIPPED [ 59%]
  821. 1102.08 s tests/primitives/test_python_interpreter.py::test_tool_default_args SKIPPED [ 59%]
  822. 1102.08 s tests/primitives/test_python_interpreter.py::test_tools_re_register_after_process_restart SKIPPED [ 59%]
  823. 1102.08 s tests/primitives/test_python_interpreter.py::test_mounts_replay_after_process_restart SKIPPED [ 59%]
  824. 1102.08 s tests/primitives/test_python_interpreter.py::test_tool_all_positional_args SKIPPED [ 59%]
  825. 1102.08 s tests/primitives/test_python_interpreter.py::test_tool_error_surfaces_as_runtime_error SKIPPED [ 59%]
  826. 1102.08 s tests/primitives/test_python_interpreter.py::test_tool_async_def_function SKIPPED [ 59%]
  827. 1102.08 s tests/primitives/test_python_interpreter.py::test_tool_async_def_raises_propagates SKIPPED [ 59%]
  828. 1102.08 s tests/primitives/test_python_interpreter.py::test_submit_with_typed_signature SKIPPED [ 59%]
  829. 1102.08 s tests/primitives/test_python_interpreter.py::test_submit_positional_args SKIPPED [ 59%]
  830. 1102.08 s tests/primitives/test_python_interpreter.py::test_submit_multi_output SKIPPED [ 59%]
  831. 1102.09 s tests/primitives/test_python_interpreter.py::test_submit_wrong_arg_count SKIPPED [ 60%]
  832. 1102.09 s tests/primitives/test_python_interpreter.py::test_extract_parameters SKIPPED [ 60%]
  833. 1102.09 s tests/primitives/test_python_interpreter.py::test_extract_parameters_complex_types SKIPPED [ 60%]
  834. 1102.09 s tests/primitives/test_python_interpreter.py::test_large_variable_injection SKIPPED [ 60%]
  835. 1102.09 s tests/primitives/test_python_interpreter.py::test_large_variable_content_integrity SKIPPED [ 60%]
  836. 1102.09 s tests/primitives/test_python_interpreter.py::test_mixed_small_and_large_variables SKIPPED [ 60%]
  837. 1102.09 s tests/primitives/test_python_interpreter.py::test_multiple_large_variables SKIPPED [ 60%]
  838. 1102.09 s tests/primitives/test_python_interpreter.py::test_large_list_variable SKIPPED [ 60%]
  839. 1102.09 s tests/primitives/test_python_interpreter.py::test_nested_sets_and_tuples SKIPPED [ 60%]
  840. 1102.09 s tests/primitives/test_python_interpreter.py::test_small_variable_not_using_filesystem SKIPPED [ 60%]
  841. 1102.09 s tests/primitives/test_python_interpreter.py::test_large_variable_threshold_boundary SKIPPED [ 60%]
  842. 1102.09 s tests/primitives/test_python_interpreter.py::test_enable_read_paths_symlink SKIPPED [ 60%]
  843. 1102.09 s tests/primitives/test_python_interpreter.py::test_enable_read_paths_multiple_files SKIPPED [ 60%]
  844. 1103.42 s tests/primitives/test_sandbox_serializable.py::TestABCConformance::test_subclass_conformance PASSED [ 61%]
  845. 1104.73 s tests/primitives/test_sandbox_serializable.py::TestABCConformance::test_incomplete_subclass_cannot_instantiate PASSED [ 61%]
  846. 1106.21 s tests/primitives/test_sandbox_serializable.py::TestABCConformance::test_structural_conformance_no_longer_accepted PASSED [ 61%]
  847. 1107.63 s tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_sandbox_setup PASSED [ 61%]
  848. 1108.96 s tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_to_sandbox_returns_bytes PASSED [ 61%]
  849. 1110.31 s tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_sandbox_assignment PASSED [ 61%]
  850. 1111.83 s tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_rlm_preview PASSED [ 61%]
  851. 1113.30 s tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_rlm_preview_truncation PASSED [ 61%]
  852. 1114.73 s tests/primitives/test_sandbox_serializable.py::TestBuildReplVariable::test_builds_variable_with_preview_from_rlm_preview PASSED [ 61%]
  853. 1116.21 s tests/primitives/test_sandbox_serializable.py::TestBuildReplVariable::test_surfaces_sandbox_setup_in_description PASSED [ 61%]
  854. 1117.55 s tests/primitives/test_sandbox_serializable.py::TestBuildReplVariable::test_passes_field_info_through PASSED [ 61%]
  855. 1118.94 s tests/primitives/test_sandbox_serializable.py::TestToReplVariableMethod::test_default_to_repl_variable PASSED [ 61%]
  856. 1120.29 s tests/primitives/test_sandbox_serializable.py::TestSignatureAnnotation::test_subclass_supports_signature_annotation PASSED [ 61%]
  857. 1121.80 s tests/propose/test_grounded_proposer.py::test_propose_instructions_for_program[None] PASSED [ 62%]
  858. 1123.18 s tests/propose/test_grounded_proposer.py::test_propose_instructions_for_program[demo_candidates1] PASSED [ 62%]
  859. 1124.54 s tests/propose/test_grounded_proposer.py::test_propose_instruction_for_predictor[None] PASSED [ 62%]
  860. 1125.92 s tests/propose/test_grounded_proposer.py::test_propose_instruction_for_predictor[demo_candidates1] PASSED [ 62%]
  861. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-input_formats-test_markdown_1-input1] SKIPPED [ 62%]
  862. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-input_formats-test_markdown_1-input2] SKIPPED [ 62%]
  863. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_many_types_1-input1] SKIPPED [ 62%]
  864. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_many_types_1-input2] SKIPPED [ 62%]
  865. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_nesting_2-input1] SKIPPED [ 62%]
  866. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_nesting_1-input1] SKIPPED [ 62%]
  867. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_nesting_1-input2] SKIPPED [ 62%]
  868. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-input_formats-test_markdown_1-input1] SKIPPED [ 62%]
  869. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-input_formats-test_markdown_1-input2] SKIPPED [ 62%]
  870. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_many_types_1-input1] SKIPPED [ 63%]
  871. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_many_types_1-input2] SKIPPED [ 63%]
  872. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_nesting_2-input1] SKIPPED [ 63%]
  873. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_nesting_1-input1] SKIPPED [ 63%]
  874. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_nesting_1-input2] SKIPPED [ 63%]
  875. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-input_formats-test_markdown_1-input1] SKIPPED [ 63%]
  876. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-input_formats-test_markdown_1-input2] SKIPPED [ 63%]
  877. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_many_types_1-input1] SKIPPED [ 63%]
  878. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_many_types_1-input2] SKIPPED [ 63%]
  879. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_nesting_2-input1] SKIPPED [ 63%]
  880. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_nesting_1-input1] SKIPPED [ 63%]
  881. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_nesting_1-input2] SKIPPED [ 63%]
  882. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-input_formats-test_markdown_1-input1] SKIPPED [ 63%]
  883. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-input_formats-test_markdown_1-input2] SKIPPED [ 63%]
  884. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_many_types_1-input1] SKIPPED [ 64%]
  885. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_many_types_1-input2] SKIPPED [ 64%]
  886. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_nesting_2-input1] SKIPPED [ 64%]
  887. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_nesting_1-input1] SKIPPED [ 64%]
  888. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_nesting_1-input2] SKIPPED [ 64%]
  889. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-input_formats-test_markdown_1-input1] SKIPPED [ 64%]
  890. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-input_formats-test_markdown_1-input2] SKIPPED [ 64%]
  891. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_many_types_1-input1] SKIPPED [ 64%]
  892. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_many_types_1-input2] SKIPPED [ 64%]
  893. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_nesting_2-input1] SKIPPED [ 64%]
  894. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_nesting_1-input1] SKIPPED [ 64%]
  895. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_nesting_1-input2] SKIPPED [ 64%]
  896. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-input_formats-test_markdown_1-input1] SKIPPED [ 64%]
  897. 1125.94 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-input_formats-test_markdown_1-input2] SKIPPED [ 65%]
  898. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_many_types_1-input1] SKIPPED [ 65%]
  899. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_many_types_1-input2] SKIPPED [ 65%]
  900. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_nesting_2-input1] SKIPPED [ 65%]
  901. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_nesting_1-input1] SKIPPED [ 65%]
  902. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_nesting_1-input2] SKIPPED [ 65%]
  903. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-input_formats-test_markdown_1-input1] SKIPPED [ 65%]
  904. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-input_formats-test_markdown_1-input2] SKIPPED [ 65%]
  905. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_many_types_1-input1] SKIPPED [ 65%]
  906. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_many_types_1-input2] SKIPPED [ 65%]
  907. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_nesting_2-input1] SKIPPED [ 65%]
  908. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_nesting_1-input1] SKIPPED [ 65%]
  909. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_nesting_1-input2] SKIPPED [ 65%]
  910. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-input_formats-test_markdown_1-input1] SKIPPED [ 66%]
  911. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-input_formats-test_markdown_1-input2] SKIPPED [ 66%]
  912. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_many_types_1-input1] SKIPPED [ 66%]
  913. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_many_types_1-input2] SKIPPED [ 66%]
  914. 1125.95 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_nesting_2-input1] SKIPPED [ 66%]
  915. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_nesting_1-input1] SKIPPED [ 66%]
  916. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_nesting_1-input2] SKIPPED [ 66%]
  917. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-input_formats-test_markdown_1-input1] SKIPPED [ 66%]
  918. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-input_formats-test_markdown_1-input2] SKIPPED [ 66%]
  919. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_many_types_1-input1] SKIPPED [ 66%]
  920. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_many_types_1-input2] SKIPPED [ 66%]
  921. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_nesting_2-input1] SKIPPED [ 66%]
  922. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_nesting_1-input1] SKIPPED [ 66%]
  923. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_nesting_1-input2] SKIPPED [ 67%]
  924. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 67%]
  925. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 67%]
  926. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 67%]
  927. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 67%]
  928. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 67%]
  929. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 67%]
  930. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 67%]
  931. 1125.96 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 67%]
  932. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 67%]
  933. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 67%]
  934. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 67%]
  935. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 67%]
  936. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 68%]
  937. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 68%]
  938. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 68%]
  939. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 68%]
  940. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 68%]
  941. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 68%]
  942. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 68%]
  943. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 68%]
  944. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 68%]
  945. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 68%]
  946. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 68%]
  947. 1125.97 s tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 68%]
  948. 1125.98 s tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 68%]
  949. 1125.98 s tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 68%]
  950. 1125.98 s tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 69%]
  951. 1125.98 s tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 69%]
  952. 1125.98 s tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-input_formats-test_markdown_1-input1] SKIPPED [ 69%]
  953. 1125.98 s tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-input_formats-test_markdown_1-input2] SKIPPED [ 69%]
  954. 1125.98 s tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_many_types_1-input1] SKIPPED [ 69%]
  955. 1125.98 s tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_many_types_1-input2] SKIPPED [ 69%]
  956. 1125.98 s tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_nesting_2-input1] SKIPPED [ 69%]
  957. 1125.98 s tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_nesting_1-input1] SKIPPED [ 69%]
  958. 1125.98 s tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_nesting_1-input2] SKIPPED [ 69%]
  959. 1125.98 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-4o] SKIPPED [ 69%]
  960. 1125.98 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-4o-mini] SKIPPED [ 69%]
  961. 1125.98 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-4-turbo] SKIPPED [ 69%]
  962. 1125.98 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-o1-preview] SKIPPED [ 69%]
  963. 1125.98 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-o1-mini] SKIPPED [ 70%]
  964. 1125.98 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[claude-3.5-sonnet] SKIPPED [ 70%]
  965. 1125.98 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[claude-3.5-haiku] SKIPPED [ 70%]
  966. 1125.99 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gemini-1.5-pro] SKIPPED [ 70%]
  967. 1125.99 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gemini-1.5-flash] SKIPPED [ 70%]
  968. 1125.99 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.1-405b-instruct] SKIPPED [ 70%]
  969. 1125.99 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.1-70b-instruct] SKIPPED [ 70%]
  970. 1125.99 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.1-8b-instruct] SKIPPED [ 70%]
  971. 1125.99 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.2-3b-instruct] SKIPPED [ 70%]
  972. 1125.99 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[deepseek-r1] SKIPPED [ 70%]
  973. 1125.99 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-Predict] SKIPPED [ 70%]
  974. 1125.99 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-ChainOfThought] SKIPPED [ 70%]
  975. 1125.99 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-mini-Predict] SKIPPED [ 70%]
  976. 1125.99 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-mini-ChainOfThought] SKIPPED [ 71%]
  977. 1125.99 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4-turbo-Predict] SKIPPED [ 71%]
  978. 1125.99 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4-turbo-ChainOfThought] SKIPPED [ 71%]
  979. 1125.99 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-preview-Predict] SKIPPED [ 71%]
  980. 1125.99 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-preview-ChainOfThought] SKIPPED [ 71%]
  981. 1125.99 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-mini-Predict] SKIPPED [ 71%]
  982. 1125.99 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-mini-ChainOfThought] SKIPPED [ 71%]
  983. 1125.99 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-sonnet-Predict] SKIPPED [ 71%]
  984. 1125.99 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-sonnet-ChainOfThought] SKIPPED [ 71%]
  985. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-haiku-Predict] SKIPPED [ 71%]
  986. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-haiku-ChainOfThought] SKIPPED [ 71%]
  987. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-pro-Predict] SKIPPED [ 71%]
  988. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-pro-ChainOfThought] SKIPPED [ 71%]
  989. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-flash-Predict] SKIPPED [ 72%]
  990. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-flash-ChainOfThought] SKIPPED [ 72%]
  991. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-405b-instruct-Predict] SKIPPED [ 72%]
  992. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-405b-instruct-ChainOfThought] SKIPPED [ 72%]
  993. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-70b-instruct-Predict] SKIPPED [ 72%]
  994. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-70b-instruct-ChainOfThought] SKIPPED [ 72%]
  995. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-8b-instruct-Predict] SKIPPED [ 72%]
  996. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-8b-instruct-ChainOfThought] SKIPPED [ 72%]
  997. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.2-3b-instruct-Predict] SKIPPED [ 72%]
  998. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.2-3b-instruct-ChainOfThought] SKIPPED [ 72%]
  999. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[deepseek-r1-Predict] SKIPPED [ 72%]
  1000. 1126.00 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[deepseek-r1-ChainOfThought] SKIPPED [ 72%]
  1001. 1126.00 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-4o] SKIPPED [ 72%]
  1002. 1126.01 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-4o-mini] SKIPPED [ 72%]
  1003. 1126.01 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-4-turbo] SKIPPED [ 73%]
  1004. 1126.01 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-o1-preview] SKIPPED [ 73%]
  1005. 1126.01 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-o1-mini] SKIPPED [ 73%]
  1006. 1126.01 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[claude-3.5-sonnet] SKIPPED [ 73%]
  1007. 1126.01 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[claude-3.5-haiku] SKIPPED [ 73%]
  1008. 1126.01 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gemini-1.5-pro] SKIPPED [ 73%]
  1009. 1126.01 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gemini-1.5-flash] SKIPPED [ 73%]
  1010. 1126.01 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.1-405b-instruct] SKIPPED [ 73%]
  1011. 1126.01 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.1-70b-instruct] SKIPPED [ 73%]
  1012. 1126.01 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.1-8b-instruct] SKIPPED [ 73%]
  1013. 1126.01 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.2-3b-instruct] SKIPPED [ 73%]
  1014. 1126.01 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[deepseek-r1] SKIPPED [ 73%]
  1015. 1126.01 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-Predict] SKIPPED [ 73%]
  1016. 1126.01 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-ChainOfThought] SKIPPED [ 74%]
  1017. 1126.01 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-mini-Predict] SKIPPED [ 74%]
  1018. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-mini-ChainOfThought] SKIPPED [ 74%]
  1019. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4-turbo-Predict] SKIPPED [ 74%]
  1020. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4-turbo-ChainOfThought] SKIPPED [ 74%]
  1021. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-preview-Predict] SKIPPED [ 74%]
  1022. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-preview-ChainOfThought] SKIPPED [ 74%]
  1023. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-mini-Predict] SKIPPED [ 74%]
  1024. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-mini-ChainOfThought] SKIPPED [ 74%]
  1025. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-sonnet-Predict] SKIPPED [ 74%]
  1026. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-sonnet-ChainOfThought] SKIPPED [ 74%]
  1027. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-haiku-Predict] SKIPPED [ 74%]
  1028. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-haiku-ChainOfThought] SKIPPED [ 74%]
  1029. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-pro-Predict] SKIPPED [ 75%]
  1030. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-pro-ChainOfThought] SKIPPED [ 75%]
  1031. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-flash-Predict] SKIPPED [ 75%]
  1032. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-flash-ChainOfThought] SKIPPED [ 75%]
  1033. 1126.02 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-405b-instruct-Predict] SKIPPED [ 75%]
  1034. 1126.03 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-405b-instruct-ChainOfThought] SKIPPED [ 75%]
  1035. 1126.03 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-70b-instruct-Predict] SKIPPED [ 75%]
  1036. 1126.03 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-70b-instruct-ChainOfThought] SKIPPED [ 75%]
  1037. 1126.03 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-8b-instruct-Predict] SKIPPED [ 75%]
  1038. 1126.03 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-8b-instruct-ChainOfThought] SKIPPED [ 75%]
  1039. 1126.03 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.2-3b-instruct-Predict] SKIPPED [ 75%]
  1040. 1126.03 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.2-3b-instruct-ChainOfThought] SKIPPED [ 75%]
  1041. 1126.03 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[deepseek-r1-Predict] SKIPPED [ 75%]
  1042. 1126.03 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[deepseek-r1-ChainOfThought] SKIPPED [ 76%]
  1043. 1127.53 s tests/retrievers/test_colbertv2.py::test_get_request_raises_on_server_error PASSED [ 76%]
  1044. 1128.91 s tests/retrievers/test_colbertv2.py::test_post_request_raises_on_server_error PASSED [ 76%]
  1045. 1130.36 s tests/retrievers/test_colbertv2.py::test_get_request_success PASSED [ 76%]
  1046. 1131.94 s tests/retrievers/test_colbertv2.py::test_post_request_success PASSED [ 76%]
  1047. 1133.37 s tests/retrievers/test_embeddings.py::test_embeddings_basic_search PASSED [ 76%]
  1048. 1134.99 s tests/retrievers/test_embeddings.py::test_embeddings_multithreaded_search PASSED [ 76%]
  1049. 1136.66 s tests/retrievers/test_embeddings.py::test_embeddings_save_load PASSED [ 76%]
  1050. 1137.97 s tests/retrievers/test_embeddings.py::test_embeddings_from_saved PASSED [ 76%]
  1051. 1139.43 s tests/retrievers/test_embeddings.py::test_embeddings_load_nonexistent_path PASSED [ 76%]
  1052. 1140.90 s tests/retrievers/test_embeddings.py::test_embeddings_with_scores_basic_search PASSED [ 76%]
  1053. 1142.59 s tests/retrievers/test_embeddings.py::test_embeddings_with_scores_save_load PASSED [ 76%]
  1054. 1143.99 s tests/signatures/test_adapter_file.py::test_file_from_local_path PASSED [ 76%]
  1055. 1145.35 s tests/signatures/test_adapter_file.py::test_file_from_path_method PASSED [ 77%]
  1056. 1146.92 s tests/signatures/test_adapter_file.py::test_file_from_path_with_custom_filename PASSED [ 77%]
  1057. 1148.26 s tests/signatures/test_adapter_file.py::test_file_from_bytes PASSED [ 77%]
  1058. 1149.63 s tests/signatures/test_adapter_file.py::test_file_from_bytes_with_filename PASSED [ 77%]
  1059. 1150.99 s tests/signatures/test_adapter_file.py::test_file_from_file_id PASSED [ 77%]
  1060. 1152.47 s tests/signatures/test_adapter_file.py::test_file_from_file_id_with_filename PASSED [ 77%]
  1061. 1153.84 s tests/signatures/test_adapter_file.py::test_file_from_dict_with_file_data PASSED [ 77%]
  1062. 1155.20 s tests/signatures/test_adapter_file.py::test_file_from_dict_with_file_id PASSED [ 77%]
  1063. 1156.72 s tests/signatures/test_adapter_file.py::test_file_format_with_file_data PASSED [ 77%]
  1064. 1158.15 s tests/signatures/test_adapter_file.py::test_file_format_with_file_id PASSED [ 77%]
  1065. 1159.54 s tests/signatures/test_adapter_file.py::test_file_repr_with_file_data PASSED [ 77%]
  1066. 1160.91 s tests/signatures/test_adapter_file.py::test_file_repr_with_file_id PASSED [ 77%]
  1067. 1162.50 s tests/signatures/test_adapter_file.py::test_file_str PASSED [ 77%]
  1068. 1163.89 s tests/signatures/test_adapter_file.py::test_encode_file_to_dict_from_path PASSED [ 77%]
  1069. 1165.30 s tests/signatures/test_adapter_file.py::test_encode_file_to_dict_from_bytes PASSED [ 78%]
  1070. 1166.80 s tests/signatures/test_adapter_file.py::test_invalid_file_string PASSED [ 78%]
  1071. 1168.13 s tests/signatures/test_adapter_file.py::test_invalid_dict PASSED [ 78%]
  1072. 1169.58 s tests/signatures/test_adapter_file.py::test_file_in_signature PASSED [ 78%]
  1073. 1170.94 s tests/signatures/test_adapter_file.py::test_file_list_in_signature PASSED [ 78%]
  1074. 1172.13 s tests/signatures/test_adapter_file.py::test_optional_file_field PASSED [ 78%]
  1075. 1172.98 s tests/signatures/test_adapter_file.py::test_save_load_file_signature PASSED [ 78%]
  1076. 1173.83 s tests/signatures/test_adapter_file.py::test_file_frozen PASSED [ 78%]
  1077. 1175.15 s tests/signatures/test_adapter_file.py::test_file_with_all_fields PASSED [ 78%]
  1078. 1176.68 s tests/signatures/test_adapter_file.py::test_file_path_not_found PASSED [ 78%]
  1079. 1178.09 s tests/signatures/test_adapter_file.py::test_file_custom_mime_type PASSED [ 78%]
  1080. 1179.29 s tests/signatures/test_adapter_file.py::test_file_from_bytes_custom_mime PASSED [ 78%]
  1081. 1180.69 s tests/signatures/test_adapter_file.py::test_file_data_uri_in_format PASSED [ 78%]
  1082. 1182.18 s tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case0] PASSED [ 79%]
  1083. 1183.54 s tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case1] PASSED [ 79%]
  1084. 1184.87 s tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case2] PASSED [ 79%]
  1085. 1186.34 s tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case3] PASSED [ 79%]
  1086. 1187.68 s tests/signatures/test_adapter_image.py::test_image_input_formats[pil_image-PIL Image] XFAIL [ 79%]
  1087. 1189.16 s tests/signatures/test_adapter_image.py::test_image_input_formats[encoded_pil_image-encoded PIL image string] XFAILon) [ 79%]
  1088. 1190.63 s tests/signatures/test_adapter_image.py::test_image_input_formats[dspy_image_download-dspy.Image with download=True] PASSED [ 79%]
  1089. 1192.24 s tests/signatures/test_adapter_image.py::test_image_input_formats[dspy_image_no_download-dspy.Image without download] PASSED [ 79%]
  1090. 1193.69 s tests/signatures/test_adapter_image.py::test_predictor_save_load PASSED [ 79%]
  1091. 1195.07 s tests/signatures/test_adapter_image.py::test_save_load_complex_default_types PASSED [ 79%]
  1092. 1196.62 s tests/signatures/test_adapter_image.py::test_save_load_complex_types[test_case0] PASSED [ 79%]
  1093. 1197.97 s tests/signatures/test_adapter_image.py::test_save_load_complex_types[test_case1] PASSED [ 79%]
  1094. 1199.34 s tests/signatures/test_adapter_image.py::test_save_load_pydantic_model PASSED [ 79%]
  1095. 1200.77 s tests/signatures/test_adapter_image.py::test_optional_image_field PASSED [ 80%]
  1096. 1202.28 s tests/signatures/test_adapter_image.py::test_pdf_url_support PASSED [ 80%]
  1097. 1203.67 s tests/signatures/test_adapter_image.py::test_different_mime_types PASSED [ 80%]
  1098. 1205.10 s tests/signatures/test_adapter_image.py::test_mime_type_from_response_headers PASSED [ 80%]
  1099. 1206.68 s tests/signatures/test_adapter_image.py::test_pdf_from_file PASSED [ 80%]
  1100. 1208.09 s tests/signatures/test_adapter_image.py::test_image_repr PASSED [ 80%]
  1101. 1209.53 s tests/signatures/test_adapter_image.py::test_from_methods_warn PASSED [ 80%]
  1102. 1210.95 s tests/signatures/test_adapter_image.py::test_invalid_string_format PASSED [ 80%]
  1103. 1212.51 s tests/signatures/test_adapter_image.py::test_pil_image_with_download_parameter PASSED [ 80%]
  1104. 1213.87 s tests/signatures/test_custom_types.py::test_basic_custom_type_resolution PASSED [ 80%]
  1105. 1215.26 s tests/signatures/test_custom_types.py::test_type_alias_for_nested_types PASSED [ 80%]
  1106. 1216.77 s tests/signatures/test_custom_types.py::test_module_level_type_resolution PASSED [ 80%]
  1107. 1218.16 s tests/signatures/test_custom_types.py::test_recommended_patterns PASSED [ 80%]
  1108. 1219.58 s tests/signatures/test_custom_types.py::test_expected_failure PASSED [ 81%]
  1109. 1220.98 s tests/signatures/test_custom_types.py::test_module_type_resolution PASSED [ 81%]
  1110. 1222.50 s tests/signatures/test_signature.py::test_field_types_and_custom_attributes PASSED [ 81%]
  1111. 1223.90 s tests/signatures/test_signature.py::test_no_input_output PASSED [ 81%]
  1112. 1225.36 s tests/signatures/test_signature.py::test_no_input_output2 PASSED [ 81%]
  1113. 1226.84 s tests/signatures/test_signature.py::test_all_fields_have_prefix PASSED [ 81%]
  1114. 1228.28 s tests/signatures/test_signature.py::test_signature_parsing PASSED [ 81%]
  1115. 1229.69 s tests/signatures/test_signature.py::test_duplicate_input_output_field_names_raise PASSED [ 81%]
  1116. 1231.19 s tests/signatures/test_signature.py::test_with_signature PASSED [ 81%]
  1117. 1232.53 s tests/signatures/test_signature.py::test_append_instructions PASSED [ 81%]
  1118. 1233.85 s tests/signatures/test_signature.py::test_append_instructions_chains PASSED [ 81%]
  1119. 1235.17 s tests/signatures/test_signature.py::test_append_instructions_preserves_default_instructions PASSED [ 81%]
  1120. 1236.65 s tests/signatures/test_signature.py::test_with_updated_field PASSED [ 81%]
  1121. 1238.01 s tests/signatures/test_signature.py::test_empty_signature PASSED [ 81%]
  1122. 1239.37 s tests/signatures/test_signature.py::test_instructions_signature PASSED [ 82%]
  1123. 1240.72 s tests/signatures/test_signature.py::test_signature_instructions PASSED [ 82%]
  1124. 1242.26 s tests/signatures/test_signature.py::test_signature_instructions_none PASSED [ 82%]
  1125. 1243.63 s tests/signatures/test_signature.py::test_signature_from_dict PASSED [ 82%]
  1126. 1245.08 s tests/signatures/test_signature.py::test_signature_equality PASSED [ 82%]
  1127. 1246.64 s tests/signatures/test_signature.py::test_signature_inequality PASSED [ 82%]
  1128. 1247.90 s tests/signatures/test_signature.py::test_equality_format PASSED [ 82%]
  1129. 1249.26 s tests/signatures/test_signature.py::test_signature_reverse PASSED [ 82%]
  1130. 1250.58 s tests/signatures/test_signature.py::test_insert_field_at_various_positions PASSED [ 82%]
  1131. 1251.95 s tests/signatures/test_signature.py::test_order_preserved_with_mixed_annotations PASSED [ 82%]
  1132. 1253.28 s tests/signatures/test_signature.py::test_infer_prefix PASSED [ 82%]
  1133. 1254.63 s tests/signatures/test_signature.py::test_insantiating PASSED [ 82%]
  1134. 1256.16 s tests/signatures/test_signature.py::test_insantiating2 PASSED [ 82%]
  1135. 1257.58 s tests/signatures/test_signature.py::test_multiline_instructions PASSED [ 83%]
  1136. 1259.04 s tests/signatures/test_signature.py::test_dump_and_load_state PASSED [ 83%]
  1137. 1260.46 s tests/signatures/test_signature.py::test_typed_signatures_basic_types PASSED [ 83%]
  1138. 1262.03 s tests/signatures/test_signature.py::test_typed_signatures_generics PASSED [ 83%]
  1139. 1263.40 s tests/signatures/test_signature.py::test_typed_signatures_unions_and_optionals PASSED [ 83%]
  1140. 1264.76 s tests/signatures/test_signature.py::test_typed_signatures_any PASSED [ 83%]
  1141. 1266.27 s tests/signatures/test_signature.py::test_typed_signatures_nested PASSED [ 83%]
  1142. 1267.68 s tests/signatures/test_signature.py::test_typed_signatures_from_dict PASSED [ 83%]
  1143. 1269.09 s tests/signatures/test_signature.py::test_typed_signatures_complex_combinations PASSED [ 83%]
  1144. 1270.37 s tests/signatures/test_signature.py::test_make_signature_from_string PASSED [ 83%]
  1145. 1271.90 s tests/signatures/test_signature.py::test_signature_field_with_constraints PASSED [ 83%]
  1146. 1273.30 s tests/signatures/test_signature.py::test_basic_custom_type PASSED [ 83%]
  1147. 1274.69 s tests/signatures/test_signature.py::test_custom_type_from_different_module PASSED [ 83%]
  1148. 1275.94 s tests/signatures/test_signature.py::test_pep604_union_type_inline PASSED [ 84%]
  1149. 1277.45 s tests/signatures/test_signature.py::test_pep604_union_type_inline_equivalence PASSED [ 84%]
  1150. 1278.89 s tests/signatures/test_signature.py::test_pep604_union_type_inline_nested PASSED [ 84%]
  1151. 1280.33 s tests/signatures/test_signature.py::test_pep604_union_type_class_nested PASSED [ 84%]
  1152. 1281.79 s tests/signatures/test_signature.py::test_pep604_union_type_class_equivalence PASSED [ 84%]
  1153. 1283.21 s tests/signatures/test_signature.py::test_pep604_union_type_insert PASSED [ 84%]
  1154. 1284.62 s tests/signatures/test_signature.py::test_pep604_union_type_with_custom_types PASSED [ 84%]
  1155. 1286.21 s tests/signatures/test_signature.py::test_signature_cloudpickle_roundtrip PASSED [ 84%]
  1156. 1287.65 s tests/signatures/test_signature.py::test_predict_cloudpickle_roundtrip PASSED [ 84%]
  1157. 1291.65 s tests/streaming/test_streaming.py::test_streamify_yields_expected_response_chunks PASSED [ 84%]
  1158. 1296.13 s tests/streaming/test_streaming.py::test_streaming_response_yields_expected_response_chunks PASSED [ 84%]
  1159. 1297.52 s tests/streaming/test_streaming.py::test_default_status_streaming PASSED [ 84%]
  1160. 1298.90 s tests/streaming/test_streaming.py::test_custom_status_streaming PASSED [ 84%]
  1161. 1300.31 s tests/streaming/test_streaming.py::test_concurrent_status_message_providers PASSED [ 85%]
  1162. 1300.31 s tests/streaming/test_streaming.py::test_stream_listener_chat_adapter SKIPPED [ 85%]
  1163. 1301.82 s tests/streaming/test_streaming.py::test_default_status_streaming_in_async_program PASSED [ 85%]
  1164. 1301.83 s tests/streaming/test_streaming.py::test_stream_listener_json_adapter SKIPPED [ 85%]
  1165. 1303.66 s tests/streaming/test_streaming.py::test_streaming_handles_space_correctly PASSED [ 85%]
  1166. 1303.68 s tests/streaming/test_streaming.py::test_sync_streaming SKIPPED (need
  1167. 1303.68 s --llm_call option to run) [ 85%]
  1168. 1305.05 s tests/streaming/test_streaming.py::test_sync_status_streaming PASSED [ 85%]
  1169. 1306.61 s tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_chat_adapter PASSED [ 85%]
  1170. 1308.07 s tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_json_adapter PASSED [ 85%]
  1171. 1309.50 s tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_chat_adapter_untokenized_stream PASSED [ 85%]
  1172. 1310.91 s tests/streaming/test_streaming.py::test_stream_listener_missing_completion_marker_chat_adapter PASSED [ 85%]
  1173. 1312.35 s tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_json_adapter_untokenized_stream PASSED [ 85%]
  1174. 1314.72 s tests/streaming/test_streaming.py::test_status_message_non_blocking PASSED [ 85%]
  1175. 1317.22 s tests/streaming/test_streaming.py::test_status_message_non_blocking_async_program PASSED [ 86%]
  1176. 1318.57 s tests/streaming/test_streaming.py::test_stream_listener_allow_reuse PASSED [ 86%]
  1177. 1319.93 s tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_xml_adapter PASSED [ 86%]
  1178. 1321.37 s tests/streaming/test_streaming.py::test_streaming_allows_custom_chunk_types PASSED [ 86%]
  1179. 1322.83 s tests/streaming/test_streaming.py::test_streaming_allows_custom_streamable_type PASSED [ 86%]
  1180. 1324.22 s tests/streaming/test_streaming.py::test_streaming_with_citations PASSED [ 86%]
  1181. 1325.58 s tests/streaming/test_streaming.py::test_chat_adapter_simple_pydantic_streaming PASSED [ 86%]
  1182. 1327.14 s tests/streaming/test_streaming.py::test_chat_adapter_with_generic_type_annotation PASSED [ 86%]
  1183. 1328.60 s tests/streaming/test_streaming.py::test_chat_adapter_nested_pydantic_streaming PASSED [ 86%]
  1184. 1329.91 s tests/streaming/test_streaming.py::test_chat_adapter_mixed_fields_streaming PASSED [ 86%]
  1185. 1331.45 s tests/streaming/test_streaming.py::test_json_adapter_simple_pydantic_streaming PASSED [ 86%]
  1186. 1333.07 s tests/streaming/test_streaming.py::test_json_adapter_bracket_balance_detection PASSED [ 86%]
  1187. 1334.72 s tests/streaming/test_streaming.py::test_json_adapter_multiple_fields_detection PASSED [ 86%]
  1188. 1336.26 s tests/streaming/test_streaming.py::test_stream_listener_could_form_end_identifier_chat_adapter PASSED [ 86%]
  1189. 1337.64 s tests/streaming/test_streaming.py::test_stream_listener_could_form_end_identifier_json_adapter PASSED [ 87%]
  1190. 1339.03 s tests/streaming/test_streaming.py::test_stream_listener_could_form_end_identifier_xml_adapter PASSED [ 87%]
  1191. 1340.45 s tests/streaming/test_streaming.py::test_streaming_reasoning_model PASSED [ 87%]
  1192. 1341.99 s tests/streaming/test_streaming.py::test_stream_listener_empty_last_chunk_chat_adapter PASSED [ 87%]
  1193. 1343.36 s tests/streaming/test_streaming.py::test_stream_listener_empty_last_chunk_json_adapter PASSED [ 87%]
  1194. 1344.75 s tests/streaming/test_streaming.py::test_streaming_reasoning_fallback PASSED [ 87%]
  1195. 1346.26 s tests/teleprompt/test_bettertogether.py::test_bettertogether_import PASSED [ 87%]
  1196. 1347.64 s tests/teleprompt/test_bettertogether.py::test_bettertogether_initialization_default PASSED [ 87%]
  1197. 1349.04 s tests/teleprompt/test_bettertogether.py::test_bettertogether_initialization_custom PASSED [ 87%]
  1198. 1350.46 s tests/teleprompt/test_bettertogether.py::test_bettertogether_initialization_invalid_optimizer PASSED [ 87%]
  1199. 1351.94 s tests/teleprompt/test_bettertogether.py::test_strategy_validation PASSED [ 87%]
  1200. 1353.32 s tests/teleprompt/test_bettertogether.py::test_compile_basic PASSED [ 87%]
  1201. 1354.70 s tests/teleprompt/test_bettertogether.py::test_trainset_validation PASSED [ 87%]
  1202. 1356.16 s tests/teleprompt/test_bettertogether.py::test_valset_ratio_validation PASSED [ 88%]
  1203. 1357.54 s tests/teleprompt/test_bettertogether.py::test_optimizer_compile_args_validation PASSED [ 88%]
  1204. 1358.87 s tests/teleprompt/test_bettertogether.py::test_student_in_optimizer_compile_args PASSED [ 88%]
  1205. 1360.26 s tests/teleprompt/test_bettertogether.py::test_compile_args_passed_to_optimizer PASSED [ 88%]
  1206. 1361.77 s tests/teleprompt/test_bettertogether.py::test_compile_args_multi_optimizer_strategy PASSED [ 88%]
  1207. 1363.16 s tests/teleprompt/test_bettertogether.py::test_compile_args_override_global_params PASSED [ 88%]
  1208. 1364.61 s tests/teleprompt/test_bettertogether.py::test_trainset_shuffling_between_steps PASSED [ 88%]
  1209. 1366.22 s tests/teleprompt/test_bettertogether.py::test_strategy_execution_order PASSED [ 88%]
  1210. 1367.72 s tests/teleprompt/test_bettertogether.py::test_lm_lifecycle_management PASSED [ 88%]
  1211. 1369.09 s tests/teleprompt/test_bettertogether.py::test_error_handling_returns_best_program PASSED [ 88%]
  1212. 1370.46 s tests/teleprompt/test_bettertogether.py::test_program_selection[test_valset0-p_optimized-With valset: returns best score (p), not latest (w)] PASSED [ 88%]
  1213. 1371.99 s tests/teleprompt/test_bettertogether.py::test_program_selection[None-w_optimized-Without valset: returns latest program (w)] PASSED [ 88%]
  1214. 1373.40 s tests/teleprompt/test_bettertogether.py::test_candidate_programs_structure PASSED [ 88%]
  1215. 1374.77 s tests/teleprompt/test_bettertogether.py::test_empty_valset_handling PASSED [ 89%]
  1216. 1376.30 s tests/teleprompt/test_bootstrap.py::test_bootstrap_initialization PASSED [ 89%]
  1217. 1377.73 s tests/teleprompt/test_bootstrap.py::test_compile_with_predict_instances PASSED [ 89%]
  1218. 1379.15 s tests/teleprompt/test_bootstrap.py::test_bootstrap_effectiveness PASSED [ 89%]
  1219. 1380.54 s tests/teleprompt/test_bootstrap.py::test_error_handling_during_bootstrap PASSED [ 89%]
  1220. 1382.06 s tests/teleprompt/test_bootstrap.py::test_validation_set_usage PASSED [ 89%]
  1221. 1383.46 s tests/teleprompt/test_bootstrap_finetune.py::test_bootstrap_finetune_initialization PASSED [ 89%]
  1222. 1384.98 s tests/teleprompt/test_bootstrap_finetune.py::test_compile_with_predict_instances PASSED [ 89%]
  1223. 1386.54 s tests/teleprompt/test_bootstrap_finetune.py::test_error_handling_missing_lm PASSED [ 89%]
  1224. 1388.02 s tests/teleprompt/test_bootstrap_trace.py::test_bootstrap_trace_data PASSED [ 89%]
  1225. 1389.43 s tests/teleprompt/test_bootstrap_trace.py::test_bootstrap_trace_data_passes_callback_metadata PASSED [ 89%]
  1226. 1390.71 s tests/teleprompt/test_copro_optimizer.py::test_signature_optimizer_initialization PASSED [ 89%]
  1227. 1392.23 s tests/teleprompt/test_copro_optimizer.py::test_signature_optimizer_optimization_process PASSED [ 89%]
  1228. 1393.63 s tests/teleprompt/test_copro_optimizer.py::test_signature_optimizer_statistics_tracking PASSED [ 90%]
  1229. 1395.12 s tests/teleprompt/test_copro_optimizer.py::test_optimization_and_output_verification PASSED [ 90%]
  1230. 1396.71 s tests/teleprompt/test_copro_optimizer.py::test_statistics_tracking_during_optimization PASSED [ 90%]
  1231. 1398.13 s tests/teleprompt/test_ensemble.py::test_ensemble_without_reduction PASSED [ 90%]
  1232. 1399.58 s tests/teleprompt/test_ensemble.py::test_ensemble_with_reduction PASSED [ 90%]
  1233. 1400.95 s tests/teleprompt/test_ensemble.py::test_ensemble_with_size_limitation PASSED [ 90%]
  1234. 1402.41 s tests/teleprompt/test_ensemble.py::test_ensemble_deterministic_behavior PASSED [ 90%]
  1235. 1403.85 s tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[None-batch0-expected_callback_metadata0] PASSED [ 90%]
  1236. 1405.26 s tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[None-batch1-expected_callback_metadata1] PASSED [ 90%]
  1237. 1406.72 s tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[1-batch2-expected_callback_metadata2] PASSED [ 90%]
  1238. 1408.20 s tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[1-batch3-expected_callback_metadata3] PASSED [ 90%]
  1239. 1409.69 s tests/teleprompt/test_gepa.py::test_basic_workflow[True] PASSED [ 90%]
  1240. 1411.26 s tests/teleprompt/test_gepa.py::test_basic_workflow[False] PASSED [ 90%]
  1241. 1412.75 s tests/teleprompt/test_gepa.py::test_workflow_with_custom_instruction_proposer_and_component_selector PASSED [ 90%]
  1242. 1414.17 s tests/teleprompt/test_gepa.py::test_metric_requires_feedback_signature PASSED [ 91%]
  1243. 1415.59 s tests/teleprompt/test_gepa.py::test_reflection_prompt_template_in_gepa_kwargs_raises PASSED [ 91%]
  1244. 1417.05 s tests/teleprompt/test_gepa.py::test_gepa_compile_with_track_usage_no_tuple_error PASSED [ 91%]
  1245. 1418.44 s tests/teleprompt/test_gepa.py::test_component_selector_functionality PASSED [ 91%]
  1246. 1419.91 s tests/teleprompt/test_gepa.py::test_component_selector_default_behavior PASSED [ 91%]
  1247. 1421.50 s tests/teleprompt/test_gepa.py::test_component_selector_string_round_robin PASSED [ 91%]
  1248. 1422.99 s tests/teleprompt/test_gepa.py::test_component_selector_string_all PASSED [ 91%]
  1249. 1424.44 s tests/teleprompt/test_gepa.py::test_component_selector_custom_random PASSED [ 91%]
  1250. 1425.92 s tests/teleprompt/test_gepa.py::test_alternating_half_component_selector PASSED [ 91%]
  1251. 1427.52 s tests/teleprompt/test_gepa.py::test_track_stats_result_structure PASSED [ 91%]
  1252. 1428.93 s tests/teleprompt/test_gepa.py::test_track_best_outputs_result_structure PASSED [ 91%]
  1253. 1430.36 s tests/teleprompt/test_gepa_instruction_proposer.py::test_reflection_lm_gets_structured_images PASSED [ 91%]
  1254. 1431.84 s tests/teleprompt/test_gepa_instruction_proposer.py::test_custom_proposer_without_reflection_lm PASSED [ 91%]
  1255. 1433.19 s tests/teleprompt/test_gepa_instruction_proposer.py::test_image_serialization_into_strings PASSED [ 92%]
  1256. 1434.56 s tests/teleprompt/test_gepa_instruction_proposer.py::test_default_proposer[True] PASSED [ 92%]
  1257. 1435.92 s tests/teleprompt/test_gepa_instruction_proposer.py::test_default_proposer[False] PASSED [ 92%]
  1258. 1437.33 s tests/teleprompt/test_grpo.py::test_grpo_dataset_shuffler PASSED [ 92%]
  1259. 1438.65 s tests/teleprompt/test_grpo.py::test_grpo_dataset_shuffler_with_num_ex_per_step_less_dataset PASSED [ 92%]
  1260. 1439.99 s tests/teleprompt/test_grpo.py::test_grpo_dataset_shuffler_with_num_ex_per_step_greater_dataset PASSED [ 92%]
  1261. 1441.47 s tests/teleprompt/test_knn_fewshot.py::test_knn_few_shot_initialization PASSED [ 92%]
  1262. 1443.35 s tests/teleprompt/test_random_search.py::test_basic_workflow PASSED [ 92%]
  1263. 1444.71 s tests/teleprompt/test_teleprompt.py::test_get_params PASSED [ 92%]
  1264. 1446.15 s tests/teleprompt/test_utils.py::test_eval_candidate_program_full_trainset PASSED [ 92%]
  1265. 1447.53 s tests/teleprompt/test_utils.py::test_eval_candidate_program_minibatch PASSED [ 92%]
  1266. 1448.87 s tests/teleprompt/test_utils.py::test_eval_candidate_program_failure PASSED [ 92%]
  1267. 1450.26 s tests/teleprompt/test_utils.py::test_create_n_fewshot_demo_sets_passes_metric_threshold_for_unshuffled PASSED [ 92%]
  1268. 1451.75 s tests/utils/test_annotation.py::test_experimental_decorator_on_function PASSED [ 93%]
  1269. 1453.06 s tests/utils/test_annotation.py::test_experimental_decorator_on_function_with_version PASSED [ 93%]
  1270. 1454.74 s tests/utils/test_annotation.py::test_experimental_decorator_on_class PASSED [ 93%]
  1271. 1456.36 s tests/utils/test_annotation.py::test_experimental_decorator_on_class_with_version PASSED [ 93%]
  1272. 1457.78 s tests/utils/test_annotation.py::test_experimental_decorator_without_docstring PASSED [ 93%]
  1273. 1459.29 s tests/utils/test_annotation.py::test_experimental_decorator_without_docstring_with_version PASSED [ 93%]
  1274. 1460.77 s tests/utils/test_annotation.py::test_experimental_decorator_with_callable_syntax PASSED [ 93%]
  1275. 1462.44 s tests/utils/test_annotation.py::test_experimental_decorator_with_version_callable_syntax PASSED [ 93%]
  1276. 1463.94 s tests/utils/test_asyncify.py::test_async_limiter PASSED [ 93%]
  1277. 1468.95 s tests/utils/test_asyncify.py::test_asyncify
  1278. 1469.91 s PASSED [ 93%]
  1279. 1471.39 s tests/utils/test_exceptions.py::test_lm_errors_are_exported_from_dspy PASSED [ 93%]
  1280. 1472.79 s tests/utils/test_exceptions.py::test_retryable_lm_errors_classification PASSED [ 93%]
  1281. 1474.19 s tests/utils/test_exceptions.py::test_lm_error_metadata PASSED [ 93%]
  1282. 1475.59 s tests/utils/test_exceptions.py::test_context_window_exceeded_error_defaults PASSED [ 94%]
  1283. 1477.07 s tests/utils/test_exceptions.py::test_context_window_exceeded_error_with_model PASSED [ 94%]
  1284. 1478.48 s tests/utils/test_exceptions.py::test_context_window_exceeded_error_with_message PASSED [ 94%]
  1285. 1479.81 s tests/utils/test_exceptions.py::test_context_window_exceeded_error_message_without_model PASSED [ 94%]
  1286. 1481.22 s tests/utils/test_exceptions.py::test_adapter_parse_error_basic PASSED [ 94%]
  1287. 1482.53 s tests/utils/test_exceptions.py::test_adapter_parse_error_with_message PASSED [ 94%]
  1288. 1483.87 s tests/utils/test_exceptions.py::test_adapter_parse_error_with_parsed_result PASSED [ 94%]
  1289. 1485.20 s tests/utils/test_lazy_import.py::test_is_available_true_for_stdlib PASSED [ 94%]
  1290. 1486.65 s tests/utils/test_lazy_import.py::test_is_available_false_for_missing PASSED [ 94%]
  1291. 1488.00 s tests/utils/test_lazy_import.py::test_is_available_does_not_import_module PASSED [ 94%]
  1292. 1489.38 s tests/utils/test_lazy_import.py::test_require_returns_lazy_module_when_present PASSED [ 94%]
  1293. 1490.78 s tests/utils/test_lazy_import.py::test_require_returns_cached_module PASSED [ 94%]
  1294. 1492.34 s tests/utils/test_lazy_import.py::test_require_is_safe_under_concurrent_first_use PASSED [ 94%]
  1295. 1493.72 s tests/utils/test_lazy_import.py::test_require_assignment_updates_materialized_module PASSED [ 95%]
  1296. 1495.14 s tests/utils/test_lazy_import.py::test_require_returns_stub_when_missing PASSED [ 95%]
  1297. 1496.73 s tests/utils/test_lazy_import.py::test_require_stub_raises_on_access_with_install_hint PASSED [ 95%]
  1298. 1498.02 s tests/utils/test_lazy_import.py::test_require_stub_uses_install_hint_for_litellm PASSED [ 95%]
  1299. 1499.40 s tests/utils/test_lazy_import.py::test_require_stub_uses_explicit_extra PASSED [ 95%]
  1300. 1500.80 s tests/utils/test_lazy_import.py::test_require_stub_falls_back_to_module_name PASSED [ 95%]
  1301. 1502.25 s tests/utils/test_lazy_import.py::test_install_hints_match_pyproject_extras PASSED [ 95%]
  1302. 1503.60 s tests/utils/test_magicattr.py::test_magicattr_get[l-value0] PASSED [ 95%]
  1303. 1504.94 s tests/utils/test_magicattr.py::test_magicattr_get[t.t.t.t.z-z] PASSED [ 95%]
  1304. 1506.45 s tests/utils/test_magicattr.py::test_magicattr_get[a[0]-0] PASSED [ 95%]
  1305. 1507.85 s tests/utils/test_magicattr.py::test_magicattr_get[a[1][0]-1] PASSED [ 95%]
  1306. 1509.23 s tests/utils/test_magicattr.py::test_magicattr_get[a[1][2]-value4] PASSED [ 95%]
  1307. 1510.61 s tests/utils/test_magicattr.py::test_magicattr_get[b["x"]-value5] PASSED [ 95%]
  1308. 1512.10 s tests/utils/test_magicattr.py::test_magicattr_get[b["x"]["y"]-y] PASSED [ 95%]
  1309. 1513.52 s tests/utils/test_magicattr.py::test_magicattr_get[b["z"]-value7] PASSED [ 96%]
  1310. 1514.96 s tests/utils/test_magicattr.py::test_magicattr_get[b["z"][1]-2] PASSED [ 96%]
  1311. 1516.49 s tests/utils/test_magicattr.py::test_magicattr_get[b["w"].z-z] PASSED [ 96%]
  1312. 1517.81 s tests/utils/test_magicattr.py::test_magicattr_get[b["w"].t.l-value10] PASSED [ 96%]
  1313. 1519.16 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z] PASSED [ 96%]
  1314. 1520.55 s tests/utils/test_magicattr.py::test_magicattr_get[l[-1]-2] PASSED [ 96%]
  1315. 1522.02 s tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.a[-1].z-z] PASSED [ 96%]
  1316. 1523.37 s tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.b["z"][0]-1] PASSED [ 96%]
  1317. 1524.74 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].t.z-z] PASSED [ 96%]
  1318. 1526.27 s tests/utils/test_magicattr.py::test_person_example PASSED [ 96%]
  1319. 1527.72 s tests/utils/test_magicattr.py::test_empty PASSED [ 96%]
  1320. 1527.72 s tests/utils/test_mcp.py::test_convert_mcp_tool SKIPPED (need --extra
  1321. 1527.73 s option to run) [ 96%]
  1322. 1529.15 s tests/utils/test_parallelizer.py::test_worker_threads_independence PASSED [ 96%]
  1323. 1530.69 s tests/utils/test_parallelizer.py::test_parallel_execution_speed PASSED [ 97%]
  1324. 1531.99 s tests/utils/test_parallelizer.py::test_max_errors_handling PASSED [ 97%]
  1325. 1533.41 s tests/utils/test_parallelizer.py::test_max_errors_not_met PASSED [ 97%]
  1326. 1534.68 s tests/utils/test_parallelizer.py::test_parallel_executor_tracks_failed_indices_and_exceptions PASSED [ 97%]
  1327. 1536.08 s tests/utils/test_parallelizer.py::test_sequential_execution_runs_on_main_thread PASSED [ 97%]
  1328. 1537.52 s tests/utils/test_parallelizer.py::test_sequential_max_errors_not_met PASSED [ 97%]
  1329. 1538.89 s tests/utils/test_parallelizer.py::test_sequential_max_errors_exceeded PASSED [ 97%]
  1330. 1540.09 s tests/utils/test_parallelizer.py::test_sequential_tracks_failed_indices_and_exceptions PASSED [ 97%]
  1331. 1541.57 s tests/utils/test_parallelizer.py::test_sequential_compare_results PASSED [ 97%]
  1332. 1542.96 s tests/utils/test_saving.py::test_save_predict PASSED [ 97%]
  1333. 1544.23 s tests/utils/test_saving.py::test_save_custom_model PASSED [ 97%]
  1334. 1545.63 s tests/utils/test_saving.py::test_save_model_with_custom_signature PASSED [ 97%]
  1335. 1545.63 s tests/utils/test_saving.py::test_save_compiled_model SKIPPED (need
  1336. 1545.63 s --extra option to run) [ 97%]
  1337. 1547.00 s tests/utils/test_saving.py::test_load_with_version_mismatch PASSED [ 98%]
  1338. 1548.35 s tests/utils/test_saving.py::test_pickle_loading_requires_explicit_permission PASSED [ 98%]
  1339. 1549.69 s tests/utils/test_saving.py::test_pkl_file_loading_requires_explicit_permission PASSED [ 98%]
  1340. 1551.03 s tests/utils/test_saving.py::test_json_file_loading_works_without_permission PASSED [ 98%]
  1341. 1552.46 s tests/utils/test_settings.py::test_basic_dspy_settings PASSED [ 98%]
  1342. 1553.66 s tests/utils/test_settings.py::test_forbid_configure_call_in_child_thread PASSED [ 98%]
  1343. 1555.00 s tests/utils/test_settings.py::test_dspy_context PASSED [ 98%]
  1344. 1556.48 s tests/utils/test_settings.py::test_dspy_context_parallel PASSED [ 98%]
  1345. 1558.84 s tests/utils/test_settings.py::test_dspy_context_with_dspy_parallel PASSED [ 98%]
  1346. 1561.19 s tests/utils/test_settings.py::test_dspy_context_with_async_task_group PASSED [ 98%]
  1347. 1562.71 s tests/utils/test_settings.py::test_dspy_configure_allowance_async PASSED [ 98%]
  1348. 1564.14 s tests/utils/test_settings.py::test_dspy_settings_save_load PASSED [ 98%]
  1349. 1565.56 s tests/utils/test_settings.py::test_dspy_settings_save_exclude_keys PASSED [ 98%]
  1350. 1567.09 s tests/utils/test_settings.py::test_settings_save_with_extra_modules PASSED [ 99%]
  1351. 1568.57 s tests/utils/test_syncify.py::test_syncify_in_place PASSED [ 99%]
  1352. 1569.95 s tests/utils/test_syncify.py::test_syncify_with_wrapper PASSED [ 99%]
  1353. 1571.46 s tests/utils/test_syncify.py::test_syncify_works_with_optimizers PASSED [ 99%]
  1354. 1576.46 s tests/utils/test_unbatchify.py::test_unbatchify_batch_size_trigger
  1355. 1577.84 s PASSED [ 99%]
  1356. 1579.54 s tests/utils/test_unbatchify.py::test_unbatchify_timeout_trigger PASSED [ 99%]
  1357. 1580.93 s tests/utils/test_usage_tracker.py::test_add_usage_entry PASSED [ 99%]
  1358. 1582.43 s tests/utils/test_usage_tracker.py::test_get_total_tokens PASSED [ 99%]
  1359. 1583.76 s tests/utils/test_usage_tracker.py::test_track_usage_with_multiple_models PASSED [ 99%]
  1360. 1585.09 s tests/utils/test_usage_tracker.py::test_track_usage_context_manager SKIPPED [ 99%]
  1361. 1586.53 s tests/utils/test_usage_tracker.py::test_merge_usage_entries_with_new_keys PASSED [ 99%]
  1362. 1587.85 s tests/utils/test_usage_tracker.py::test_merge_usage_entries_with_none_values PASSED [ 99%]
  1363. 1589.19 s tests/utils/test_usage_tracker.py::test_merge_usage_entries_with_pydantic_models PASSED [ 99%]
  1364. 1590.59 s tests/utils/test_usage_tracker.py::test_parallel_executor_with_usage_tracker PASSED [100%]
  1365. 1590.59 s
  1366. 1590.59 s =============================== warnings summary ===============================
  1367. 1590.59 s tests/clients/test_lm.py::test_chat_lms_can_be_queried
  1368. 1590.59 s tests/clients/test_lm.py::test_lm_calls_support_callables
  1369. 1590.59 s tests/clients/test_lm.py::test_lm_calls_support_pydantic_models
  1370. 1590.59 s /nix/store/rn0r480kscbk8159qi3qjflil71kqglq-dspy-test-env/lib/python3.13/site-packages/pydantic/main.py:464: UserWarning: Pydantic serializer warnings:
  1371. 1590.59 s PydanticSerializationUnexpectedValue(Expected 9 fields but got 5: Expected `Message` - serialized value may not be as expected [field_name='choices', input_value=Message(content='Hi!', ro...ields={'refusal': None}), input_type=Message])
  1372. 1590.59 s PydanticSerializationUnexpectedValue(Expected `StreamingChoices` - serialized value may not be as expected [field_name='choices', input_value=Choices(finish_reason='st...elds={'refusal': None})), input_type=Choices])
  1373. 1590.59 s return self.__pydantic_serializer__.to_python(
  1374. 1590.59 s
  1375. 1590.59 s tests/predict/test_multi_chain_comparison.py::test_basic_example
  1376. 1590.59 s tests/predict/test_multi_chain_comparison.py::test_basic_example
  1377. 1590.59 s tests/predict/test_multi_chain_comparison.py::test_basic_example
  1378. 1590.59 s /tmp/nix-ci-worker/stable-pwd/repo/dspy/predict/multi_chain_comparison.py:19: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version.
  1379. 1590.59 s InputField(
  1380. 1590.59 s
  1381. 1590.59 s tests/predict/test_multi_chain_comparison.py::test_basic_example
  1382. 1590.59 s /tmp/nix-ci-worker/stable-pwd/repo/dspy/predict/multi_chain_comparison.py:27: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version.
  1383. 1590.59 s OutputField(
  1384. 1590.59 s
  1385. 1590.59 s tests/predict/test_predict.py::test_signature_fields_after_dump_and_load_state
  1386. 1590.59 s /tmp/nix-ci-worker/stable-pwd/repo/tests/predict/test_predict.py:299: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version.
  1387. 1590.59 s sentiment = dspy.OutputField(desc="I am a malicious output!", prefix="I am a prefix!")
  1388. 1590.59 s
  1389. 1590.59 s tests/signatures/test_signature.py::test_all_fields_have_prefix
  1390. 1590.59 s /tmp/nix-ci-worker/stable-pwd/repo/tests/signatures/test_signature.py:46: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version.
  1391. 1590.59 s input = InputField(prefix="Modified:")
  1392. 1590.59 s
  1393. 1590.59 s tests/signatures/test_signature.py::test_equality_format
  1394. 1590.59 s /tmp/nix-ci-worker/stable-pwd/repo/tests/signatures/test_signature.py:157: DeprecationWarning: The 'format' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version.
  1395. 1590.59 s input = InputField(format=lambda x: x)
  1396. 1590.59 s
  1397. 1590.59 s tests/streaming/test_streaming.py::test_streamify_yields_expected_response_chunks
  1398. 1590.59 s tests/streaming/test_streaming.py::test_streaming_response_yields_expected_response_chunks
  1399. 1590.59 s /nix/store/rn0r480kscbk8159qi3qjflil71kqglq-dspy-test-env/lib/python3.13/site-packages/pydantic/main.py:464: UserWarning: Pydantic serializer warnings:
  1400. 1590.59 s PydanticSerializationUnexpectedValue(Expected 9 fields but got 5: Expected `Message` - serialized value may not be as expected [field_name='choices', input_value=Message(content='{"output...er_specific_fields=None), input_type=Message])
  1401. 1590.59 s PydanticSerializationUnexpectedValue(Expected `StreamingChoices` - serialized value may not be as expected [field_name='choices', input_value=Choices(finish_reason='st...r_specific_fields=None)), input_type=Choices])
  1402. 1590.59 s return self.__pydantic_serializer__.to_python(
  1403. 1590.59 s
  1404. 1590.59 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z]
  1405. 1590.59 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z]
  1406. 1590.59 s tests/utils/test_magicattr.py::test_magicattr_get[l[-1]-2]
  1407. 1590.59 s tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.a[-1].z-z]
  1408. 1590.59 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].t.z-z]
  1409. 1590.59 s tests/utils/test_magicattr.py::test_person_example
  1410. 1590.59 s tests/utils/test_magicattr.py::test_person_example
  1411. 1590.59 s tests/utils/test_magicattr.py::test_person_example
  1412. 1590.59 s tests/utils/test_magicattr.py::test_person_example
  1413. 1590.59 s /tmp/nix-ci-worker/stable-pwd/repo/dspy/utils/magicattr.py:99: DeprecationWarning: ast.Num is deprecated and will be removed in Python 3.14; use ast.Constant instead
  1414. 1590.59 s if hasattr(ast, "Num") and isinstance(node, ast.Num):
  1415. 1590.59 s
  1416. 1590.59 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z]
  1417. 1590.59 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z]
  1418. 1590.59 s tests/utils/test_magicattr.py::test_magicattr_get[l[-1]-2]
  1419. 1590.59 s tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.a[-1].z-z]
  1420. 1590.59 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].t.z-z]
  1421. 1590.59 s tests/utils/test_magicattr.py::test_person_example
  1422. 1590.59 s tests/utils/test_magicattr.py::test_person_example
  1423. 1590.59 s tests/utils/test_magicattr.py::test_person_example
  1424. 1590.59 s tests/utils/test_magicattr.py::test_person_example
  1425. 1590.59 s /tmp/nix-ci-worker/stable-pwd/repo/dspy/utils/magicattr.py:102: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
  1426. 1590.59 s elif hasattr(ast, "Str") and isinstance(node, ast.Str):
  1427. 1590.59 s
  1428. 1590.59 s tests/utils/test_settings.py::test_dspy_configure_allowance_async
  1429. 1590.59 s /tmp/nix-ci-worker/stable-pwd/repo/tests/utils/test_settings.py:194: RuntimeWarning: coroutine 'test_dspy_configure_allowance_async.<locals>.foo4' was never awaited
  1430. 1590.59 s foo4()
  1431. 1590.59 s Enable tracemalloc to get traceback where the object was allocated.
  1432. 1590.59 s See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
  1433. 1590.59 s
  1434. 1590.59 s -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
  1435. 1590.59 s ==== 991 passed, 330 skipped, 2 xfailed, 31 warnings in 1580.68s (0:26:20) =====
  1436. 1592.30 s Run succeeded.
Tail Raw Logs