test packages.x86_64-linux.impure-test

Reproduce this run
  1. 0.12 s $ /nix/store/9ypz3flqsrl5xl495mm8h645gadjsxi1-coreutils-9.11/bin/timeout --kill-after=15s 7200s /nix/store/23zk8sgzamrbmj1fday15szf7f2279s8-nix-2.34.7/bin/nix --extra-experimental-features nix-command --extra-experimental-features flakes flake clone git+https://github.com/NorfairKing/dspy --dest /tmp/nix-ci-worker/stable-pwd/repo/
  2. 0.14 s Cloning into '/tmp/nix-ci-worker/stable-pwd/repo'...
  3. 0.61 s remote: Enumerating objects: 54719, done.
  4. 0.63 s remote: Counting objects: 1% (1/87) remote: Counting objects: 2% (2/87) remote: Counting objects: 3% (3/87) remote: Counting objects: 4% (4/87) remote: Counting objects: 5% (5/87) remote: Counting objects: 6% (6/87) remote: Counting objects: 8% (7/87) remote: Counting objects: 9% (8/87) remote: Counting objects: 10% (9/87) remote: Counting objects: 11% (10/87) remote: Counting objects: 12% (11/87) remote: Counting objects: 13% (12/87) remote: Counting objects: 14% (13/87) remote: Counting objects: 16% (14/87) remote: Counting objects: 17% (15/87) remote: Counting objects: 18% (16/87) remote: Counting objects: 19% (17/87) remote: Counting objects: 20% (18/87) remote: Counting objects: 21% (19/87) remote: Counting objects: 22% (20/87) remote: Counting objects: 24% (21/87) remote: Counting objects: 25% (22/87) remote: Counting objects: 26% (23/87) remote: Counting objects: 27% (24/87) remote: Counting objects: 28% (25/87) remote: Counting objects: 29% (26/87) remote: Counting objects: 31% (27/87) remote: Counting objects: 32% (28/87) remote: Counting objects: 33% (29/87) remote: Counting objects: 34% (30/87) remote: Counting objects: 35% (31/87) remote: Counting objects: 36% (32/87) remote: Counting objects: 37% (33/87) remote: Counting objects: 39% (34/87) remote: Counting objects: 40% (35/87) remote: Counting objects: 41% (36/87) remote: Counting objects: 42% (37/87) remote: Counting objects: 43% (38/87) remote: Counting objects: 44% (39/87) remote: Counting objects: 45% (40/87) remote: Counting objects: 47% (41/87) remote: Counting objects: 48% (42/87) remote: Counting objects: 49% (43/87) remote: Counting objects: 50% (44/87) remote: Counting objects: 51% (45/87) remote: Counting objects: 52% (46/87) remote: Counting objects: 54% (47/87) remote: Counting objects: 55% (48/87) remote: Counting objects: 56% (49/87) remote: Counting objects: 57% (50/87) remote: Counting objects: 58% (51/87) remote: Counting objects: 59% (52/87) remote: Counting objects: 60% (53/87) remote: Counting objects: 62% (54/87) remote: Counting objects: 63% (55/87) remote: Counting objects: 64% (56/87) remote: Counting objects: 65% (57/87) remote: Counting objects: 66% (58/87) remote: Counting objects: 67% (59/87) remote: Counting objects: 68% (60/87) remote: Counting objects: 70% (61/87) remote: Counting objects: 71% (62/87) remote: Counting objects: 72% (63/87) remote: Counting objects: 73% (64/87) remote: Counting objects: 74% (65/87) remote: Counting objects: 75% (66/87) remote: Counting objects: 77% (67/87) remote: Counting objects: 78% (68/87) remote: Counting objects: 79% (69/87) remote: Counting objects: 80% (70/87) remote: Counting objects: 81% (71/87) remote: Counting objects: 82% (72/87) remote: Counting objects: 83% (73/87) remote: Counting objects: 85% (74/87) remote: Counting objects: 86% (75/87) remote: Counting objects: 87% (76/87) remote: Counting objects: 88% (77/87) remote: Counting objects: 89% (78/87) remote: Counting objects: 90% (79/87) remote: Counting objects: 91% (80/87) remote: Counting objects: 93% (81/87) remote: Counting objects: 94% (82/87) remote: Counting objects: 95% (83/87) remote: Counting objects: 96% (84/87) remote: Counting objects: 97% (85/87) remote: Counting objects: 98% (86/87) remote: Counting objects: 100% (87/87) remote: Counting objects: 100% (87/87), done.
  5. 0.64 s remote: Compressing objects: 1% (1/56) remote: Compressing objects: 3% (2/56) remote: Compressing objects: 5% (3/56) remote: Compressing objects: 7% (4/56) remote: Compressing objects: 8% (5/56) remote: Compressing objects: 10% (6/56) remote: Compressing objects: 12% (7/56) remote: Compressing objects: 14% (8/56) remote: Compressing objects: 16% (9/56) remote: Compressing objects: 17% (10/56) remote: Compressing objects: 19% (11/56) remote: Compressing objects: 21% (12/56) remote: Compressing objects: 23% (13/56) remote: Compressing objects: 25% (14/56) remote: Compressing objects: 26% (15/56) remote: Compressing objects: 28% (16/56) remote: Compressing objects: 30% (17/56) remote: Compressing objects: 32% (18/56) remote: Compressing objects: 33% (19/56) remote: Compressing objects: 35% (20/56) remote: Compressing objects: 37% (21/56) remote: Compressing objects: 39% (22/56) remote: Compressing objects: 41% (23/56) remote: Compressing objects: 42% (24/56) remote: Compressing objects: 44% (25/56) remote: Compressing objects: 46% (26/56) remote: Compressing objects: 48% (27/56) remote: Compressing objects: 50% (28/56) remote: Compressing objects: 51% (29/56) remote: Compressing objects: 53% (30/56) remote: Compressing objects: 55% (31/56) remote: Compressing objects: 57% (32/56) remote: Compressing objects: 58% (33/56) remote: Compressing objects: 60% (34/56) remote: Compressing objects: 62% (35/56) remote: Compressing objects: 64% (36/56) remote: Compressing objects: 66% (37/56) remote: Compressing objects: 67% (38/56) remote: Compressing objects: 69% (39/56) remote: Compressing objects: 71% (40/56) remote: Compressing objects: 73% (41/56) remote: Compressing objects: 75% (42/56) remote: Compressing objects: 76% (43/56) remote: Compressing objects: 78% (44/56) remote: Compressing objects: 80% (45/56) remote: Compressing objects: 82% (46/56) remote: Compressing objects: 83% (47/56) remote: Compressing objects: 85% (48/56) remote: Compressing objects: 87% (49/56) remote: Compressing objects: 89% (50/56) remote: Compressing objects: 91% (51/56) remote: Compressing objects: 92% (52/56) remote: Compressing objects: 94% (53/56) remote: Compressing objects: 96% (54/56) remote: Compressing objects: 98% (55/56) remote: Compressing objects: 100% (56/56) remote: Compressing objects: 100% (56/56), done.
  6. 5.63 s Receiving objects: 0% (1/54719) Receiving objects: 1% (548/54719) Receiving objects: 2% (1095/54719) Receiving objects: 3% (1642/54719) Receiving objects: 4% (2189/54719) Receiving objects: 5% (2736/54719) Receiving objects: 6% (3284/54719) Receiving objects: 7% (3831/54719) Receiving objects: 8% (4378/54719) Receiving objects: 9% (4925/54719), 16.81 MiB | 33.41 MiB/s Receiving objects: 10% (5472/54719), 16.81 MiB | 33.41 MiB/s Receiving objects: 11% (6020/54719), 16.81 MiB | 33.41 MiB/s Receiving objects: 12% (6567/54719), 16.81 MiB | 33.41 MiB/s Receiving objects: 13% (7114/54719), 16.81 MiB | 33.41 MiB/s Receiving objects: 13% (7545/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 14% (7661/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 15% (8208/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 16% (8756/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 17% (9303/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 18% (9850/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 19% (10397/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 20% (10944/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 21% (11491/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 22% (12039/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 23% (12586/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 24% (13133/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 25% (13680/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 26% (14227/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 27% (14775/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 28% (15322/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 29% (15869/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 30% (16416/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 31% (16963/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 32% (17511/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 33% (18058/54719), 32.95 MiB | 32.56 MiB/s Receiving objects: 34% (18605/54719), 50.15 MiB | 33.15 MiB/s Receiving objects: 35% (19152/54719), 50.15 MiB | 33.15 MiB/s Receiving objects: 36% (19699/54719), 50.15 MiB | 33.15 MiB/s Receiving objects: 37% (20247/54719), 50.15 MiB | 33.15 MiB/s Receiving objects: 38% (20794/54719), 50.15 MiB | 33.15 MiB/s Receiving objects: 39% (21341/54719), 50.15 MiB | 33.15 MiB/s Receiving objects: 39% (21872/54719), 66.56 MiB | 33.03 MiB/s Receiving objects: 40% (21888/54719), 66.56 MiB | 33.03 MiB/s Receiving objects: 41% (22435/54719), 66.56 MiB | 33.03 MiB/s Receiving objects: 42% (22982/54719), 66.56 MiB | 33.03 MiB/s Receiving objects: 43% (23530/54719), 66.56 MiB | 33.03 MiB/s Receiving objects: 44% (24077/54719), 83.53 MiB | 33.21 MiB/s Receiving objects: 45% (24624/54719), 83.53 MiB | 33.21 MiB/s Receiving objects: 46% (25171/54719), 83.53 MiB | 33.21 MiB/s Receiving objects: 47% (25718/54719), 83.53 MiB | 33.21 MiB/s Receiving objects: 48% (26266/54719), 83.53 MiB | 33.21 MiB/s Receiving objects: 48% (26398/54719), 101.66 MiB | 33.72 MiB/s Receiving objects: 48% (26447/54719), 118.03 MiB | 33.56 MiB/s Receiving objects: 49% (26813/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 50% (27360/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 51% (27907/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 52% (28454/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 53% (29002/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 54% (29549/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 55% (30096/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 56% (30643/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 57% (31190/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 58% (31738/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 59% (32285/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 60% (32832/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 61% (33379/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 62% (33926/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 63% (34473/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 64% (35021/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 65% (35568/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 66% (36115/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 67% (36662/54719), 135.11 MiB | 33.64 MiB/s Receiving objects: 68% (37209/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 69% (37757/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 70% (38304/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 71% (38851/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 72% (39398/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 73% (39945/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 74% (40493/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 75% (41040/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 76% (41587/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 77% (42134/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 78% (42681/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 79% (43229/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 80% (43776/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 81% (44323/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 82% (44870/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 83% (45417/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 84% (45964/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 85% (46512/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 86% (47059/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 87% (47606/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 88% (48153/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 89% (48700/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 90% (49248/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 91% (49795/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 92% (50342/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 93% (50889/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 94% (51436/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 95% (51984/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 96% (52531/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 97% (53078/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 98% (53625/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 99% (54172/54719), 152.18 MiB | 33.68 MiB/s remote: Total 54719 (delta 50), reused 38 (delta 31), pack-reused 54632 (from 2)
  7. 5.71 s Receiving objects: 100% (54719/54719), 152.18 MiB | 33.68 MiB/s Receiving objects: 100% (54719/54719), 168.30 MiB | 33.72 MiB/s, done.
  8. 5.91 s Resolving deltas: 0% (0/26548) Resolving deltas: 1% (267/26548) Resolving deltas: 2% (531/26548) Resolving deltas: 3% (798/26548) Resolving deltas: 4% (1062/26548) Resolving deltas: 5% (1328/26548) Resolving deltas: 6% (1593/26548) Resolving deltas: 7% (1859/26548) Resolving deltas: 8% (2124/26548) Resolving deltas: 9% (2392/26548) Resolving deltas: 10% (2656/26548) Resolving deltas: 11% (2921/26548) Resolving deltas: 12% (3186/26548) Resolving deltas: 13% (3452/26548) Resolving deltas: 14% (3717/26548) Resolving deltas: 15% (3984/26548) Resolving deltas: 16% (4250/26548) Resolving deltas: 17% (4514/26548) Resolving deltas: 18% (4779/26548) Resolving deltas: 19% (5045/26548) Resolving deltas: 20% (5310/26548) Resolving deltas: 21% (5576/26548) Resolving deltas: 22% (5841/26548) Resolving deltas: 23% (6107/26548) Resolving deltas: 24% (6372/26548) Resolving deltas: 25% (6637/26548) Resolving deltas: 26% (6903/26548) Resolving deltas: 27% (7168/26548) Resolving deltas: 28% (7434/26548) Resolving deltas: 29% (7699/26548) Resolving deltas: 30% (7966/26548) Resolving deltas: 31% (8230/26548) Resolving deltas: 32% (8497/26548) Resolving deltas: 33% (8761/26548) Resolving deltas: 34% (9027/26548) Resolving deltas: 35% (9293/26548) Resolving deltas: 36% (9558/26548) Resolving deltas: 37% (9824/26548) Resolving deltas: 38% (10090/26548) Resolving deltas: 39% (10354/26548) Resolving deltas: 40% (10620/26548) Resolving deltas: 41% (10885/26548) Resolving deltas: 42% (11151/26548) Resolving deltas: 43% (11416/26548) Resolving deltas: 44% (11682/26548) Resolving deltas: 45% (11948/26548) Resolving deltas: 46% (12213/26548) Resolving deltas: 47% (12478/26548) Resolving deltas: 48% (12744/26548) Resolving deltas: 49% (13009/26548) Resolving deltas: 50% (13275/26548) Resolving deltas: 51% (13540/26548) Resolving deltas: 52% (13805/26548) Resolving deltas: 53% (14071/26548) Resolving deltas: 54% (14337/26548) Resolving deltas: 55% (14602/26548) Resolving deltas: 56% (14867/26548) Resolving deltas: 57% (15134/26548) Resolving deltas: 58% (15398/26548) Resolving deltas: 59% (15665/26548) Resolving deltas: 60% (15929/26548) Resolving deltas: 61% (16195/26548) Resolving deltas: 62% (16460/26548) Resolving deltas: 63% (16726/26548) Resolving deltas: 64% (16991/26548) Resolving deltas: 65% (17257/26548) Resolving deltas: 66% (17522/26548) Resolving deltas: 67% (17788/26548) Resolving deltas: 68% (18053/26548) Resolving deltas: 69% (18321/26548) Resolving deltas: 70% (18584/26548) Resolving deltas: 71% (18850/26548) Resolving deltas: 72% (19115/26548) Resolving deltas: 73% (19381/26548) Resolving deltas: 74% (19646/26548) Resolving deltas: 75% (19911/26548) Resolving deltas: 76% (20177/26548) Resolving deltas: 77% (20442/26548) Resolving deltas: 78% (20708/26548) Resolving deltas: 79% (20973/26548) Resolving deltas: 80% (21239/26548) Resolving deltas: 81% (21504/26548) Resolving deltas: 82% (21770/26548) Resolving deltas: 83% (22035/26548) Resolving deltas: 84% (22301/26548) Resolving deltas: 85% (22566/26548) Resolving deltas: 86% (22832/26548) Resolving deltas: 87% (23097/26548) Resolving deltas: 88% (23363/26548) Resolving deltas: 89% (23628/26548) Resolving deltas: 90% (23894/26548) Resolving deltas: 91% (24161/26548) Resolving deltas: 92% (24425/26548) Resolving deltas: 93% (24690/26548) Resolving deltas: 94% (24958/26548) Resolving deltas: 95% (25221/26548) Resolving deltas: 96% (25487/26548) Resolving deltas: 97% (25752/26548) Resolving deltas: 98% (26018/26548) Resolving deltas: 99% (26283/26548) Resolving deltas: 100% (26548/26548) Resolving deltas: 100% (26548/26548), done.
  9. 6.66 s Fetching commit 453ecf257a343742170b3e1cef20271508c3e6a4 for your-first-flake ...
  10. 6.66 s $ /nix/store/bcnisk3ydfgv26v2gw3zlky24g00yww2-git-2.54.0/bin/git fetch origin 453ecf257a343742170b3e1cef20271508c3e6a4
  11. 7.03 s From https://github.com/NorfairKing/dspy
  12. 7.03 s * branch 453ecf257a343742170b3e1cef20271508c3e6a4 -> FETCH_HEAD
  13. 7.04 s $ /nix/store/9ypz3flqsrl5xl495mm8h645gadjsxi1-coreutils-9.11/bin/timeout --kill-after=15s 7200s /nix/store/bcnisk3ydfgv26v2gw3zlky24g00yww2-git-2.54.0/bin/git -c advice.detachedHead=false checkout 453ecf257a343742170b3e1cef20271508c3e6a4
  14. 7.10 s HEAD is now at 453ecf25 Add coreutils to impure-test runtime inputs
  15. 7.11 s $ /nix/store/9ypz3flqsrl5xl495mm8h645gadjsxi1-coreutils-9.11/bin/timeout --kill-after=15s 7200s /nix/store/23zk8sgzamrbmj1fday15szf7f2279s8-nix-2.34.7/bin/nix --extra-experimental-features nix-command --extra-experimental-features flakes --log-format internal-json run .#packages.x86_64-linux.impure-test --print-build-logs
  16. 8.75 s
  17. 10.76 s ============================= test session starts ==============================
  18. 10.79 s platform linux -- Python 3.13.13, pytest-8.4.2, pluggy-1.6.0 -- /nix/store/rn0r480kscbk8159qi3qjflil71kqglq-dspy-test-env/bin/python3.13
  19. 10.79 s rootdir: /tmp/nix-ci-worker/stable-pwd/repo
  20. 10.79 s configfile: pyproject.toml
  21. 10.79 s plugins: anyio-4.13.0, typeguard-4.5.1, asyncio-1.3.0, mock-3.14.1
  22. 10.79 s asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
  23. 15.79 s collecting ...
  24. 18.87 s collecting 6 items collecting 288 items collecting 670 items collecting 1156 items collected 1322 items / 1 skipped
  25. 18.87 s
  26. 23.88 s tests/adapters/test_adapter_utils.py::test_parse_value_str_annotation
  27. 24.35 s PASSED [ 0%]
  28. 27.61 s tests/adapters/test_adapter_utils.py::test_parse_value_pydantic_types PASSED [ 0%]
  29. 30.58 s tests/adapters/test_adapter_utils.py::test_parse_value_basic_types PASSED [ 0%]
  30. 33.58 s tests/adapters/test_adapter_utils.py::test_parse_value_literal PASSED [ 0%]
  31. 36.55 s tests/adapters/test_adapter_utils.py::test_parse_value_union PASSED [ 0%]
  32. 38.85 s tests/adapters/test_adapter_utils.py::test_parse_value_json_repair PASSED [ 0%]
  33. 42.36 s tests/adapters/test_audio.py::test_normalize_audio_format[wav-wav] PASSED [ 0%]
  34. 45.75 s tests/adapters/test_audio.py::test_normalize_audio_format[mp3-mp3] PASSED [ 0%]
  35. 48.83 s tests/adapters/test_audio.py::test_normalize_audio_format[x-wav-wav] PASSED [ 0%]
  36. 51.63 s tests/adapters/test_audio.py::test_normalize_audio_format[x-mp3-mp3] PASSED [ 0%]
  37. 54.40 s tests/adapters/test_audio.py::test_normalize_audio_format[x-flac-flac] PASSED [ 0%]
  38. 57.16 s tests/adapters/test_audio.py::test_normalize_audio_format[my-x-format-my-x-format] PASSED [ 0%]
  39. 59.43 s tests/adapters/test_audio.py::test_normalize_audio_format[x-my-format-my-format] PASSED [ 0%]
  40. 61.40 s tests/adapters/test_audio.py::test_normalize_audio_format[-] PASSED [ 1%]
  41. 63.27 s tests/adapters/test_audio.py::test_normalize_audio_format[x--] PASSED [ 1%]
  42. 65.08 s tests/adapters/test_baml_adapter.py::test_baml_adapter_format_exact_messages_for_simple_signature_with_demo PASSED [ 1%]
  43. 67.26 s tests/adapters/test_baml_adapter.py::test_baml_adapter_format_exact_messages_with_nested_output PASSED [ 1%]
  44. 69.30 s tests/adapters/test_baml_adapter.py::test_baml_adapter_basic_schema_generation PASSED [ 1%]
  45. 71.35 s tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_optional_fields PASSED [ 1%]
  46. 73.66 s tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_primitive_types PASSED [ 1%]
  47. 76.26 s tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_lists_with_bracket_notation PASSED [ 1%]
  48. 79.02 s tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_complex_nested_models PASSED [ 1%]
  49. 81.91 s tests/adapters/test_baml_adapter.py::test_baml_adapter_raise_error_on_circular_references PASSED [ 1%]
  50. 85.50 s tests/adapters/test_baml_adapter.py::test_baml_adapter_formats_pydantic_inputs_as_clean_json PASSED [ 1%]
  51. 88.49 s tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_mixed_input_types PASSED [ 1%]
  52. 91.57 s tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_schema_generation_errors_gracefully PASSED [ 1%]
  53. 94.54 s tests/adapters/test_baml_adapter.py::test_baml_adapter_raises_on_missing_fields PASSED [ 2%]
  54. 98.60 s tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_type_casting_errors PASSED [ 2%]
  55. 102.21 s tests/adapters/test_baml_adapter.py::test_baml_adapter_with_images PASSED [ 2%]
  56. 104.32 s tests/adapters/test_baml_adapter.py::test_baml_adapter_with_tools PASSED [ 2%]
  57. 106.69 s tests/adapters/test_baml_adapter.py::test_baml_adapter_with_code PASSED [ 2%]
  58. 108.74 s tests/adapters/test_baml_adapter.py::test_baml_adapter_with_conversation_history PASSED [ 2%]
  59. 110.70 s tests/adapters/test_baml_adapter.py::test_baml_vs_json_adapter_token_efficiency PASSED [ 2%]
  60. 113.02 s tests/adapters/test_baml_adapter.py::test_baml_vs_json_adapter_functional_compatibility PASSED [ 2%]
  61. 115.72 s tests/adapters/test_baml_adapter.py::test_baml_adapter_async_functionality PASSED [ 2%]
  62. 118.62 s tests/adapters/test_baml_adapter.py::test_baml_adapter_with_field_aliases PASSED [ 2%]
  63. 121.01 s tests/adapters/test_baml_adapter.py::test_baml_adapter_field_alias_without_description PASSED [ 2%]
  64. 123.15 s tests/adapters/test_baml_adapter.py::test_baml_adapter_multiple_pydantic_input_fields PASSED [ 2%]
  65. 125.12 s tests/adapters/test_base_type.py::test_basic_extract_custom_type_from_annotation PASSED [ 2%]
  66. 127.18 s tests/adapters/test_base_type.py::test_extract_custom_type_from_annotation_with_nested_type PASSED [ 3%]
  67. 129.52 s tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-two-Literal['one', 'two', 'three"']-Literal['four', 'five', 'six"']] PASSED [ 3%]
  68. 131.88 s tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-she's here-Literal["she's here", 'okay', 'test']-Literal['done', "maybe'soon", 'later']] PASSED [ 3%]
  69. 134.79 s tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-another-Literal['both"and\\'', 'another']-Literal['yet"another\\'', 'plain']] PASSED [ 3%]
  70. 139.13 s tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-foo-Literal['foo', 'bar']-Literal['baz', 'qux']] PASSED [ 3%]
  71. 143.07 s tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-bar-Literal[1, 'bar']-Literal[True, 3, 'foo']] PASSED [ 3%]
  72. 146.88 s tests/adapters/test_chat_adapter.py::test_chat_adapter_sync_call PASSED [ 3%]
  73. 151.89 s tests/adapters/test_chat_adapter.py::test_chat_adapter_async_call
  74. 151.97 s PASSED [ 3%]
  75. 155.78 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_for_simple_signature PASSED [ 3%]
  76. 158.29 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_demo_and_typed_outputs PASSED [ 3%]
  77. 160.76 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_nested_pydantic_models PASSED [ 3%]
  78. 162.80 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_incomplete_demo PASSED [ 3%]
  79. 165.12 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_history PASSED [ 3%]
  80. 167.47 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_list_value_for_string_input PASSED [ 4%]
  81. 170.53 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_literal_output PASSED [ 4%]
  82. 173.28 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_multimodal_custom_type_inputs PASSED [ 4%]
  83. 175.28 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_history_demo_pydantic_tools_and_image PASSED [ 4%]
  84. 176.90 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_base_custom_type_input PASSED [ 4%]
  85. 178.58 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_citations_output_demo PASSED [ 4%]
  86. 180.69 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_and_lm_kwargs_with_native_citations PASSED [ 4%]
  87. 182.85 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_preserves_passthrough_lm_kwargs PASSED [ 4%]
  88. 184.85 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_and_lm_kwargs_with_native_reasoning PASSED [ 4%]
  89. 186.97 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_calling_still_enables_native_reasoning PASSED [ 4%]
  90. 189.27 s tests/adapters/test_chat_adapter.py::test_chat_adapter_nonnative_strips_native_tool_kwargs PASSED [ 4%]
  91. 191.70 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_reasoning_and_code_outputs PASSED [ 4%]
  92. 194.46 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_and_lm_kwargs_with_native_tool_calling PASSED [ 4%]
  93. 197.22 s tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_can_request_parallel_tool_calls[adapter0] PASSED [ 4%]
  94. 200.41 s tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_can_request_parallel_tool_calls[adapter1] PASSED [ 5%]
  95. 203.34 s tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_can_request_parallel_tool_calls[adapter2] PASSED [ 5%]
  96. 206.17 s tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_respects_lm_kwargs_parallel_tool_call_override PASSED [ 5%]
  97. 209.49 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_replay PASSED [ 5%]
  98. 213.76 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_replays_parallel_tool_results PASSED [ 5%]
  99. 216.87 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_empty_user_message PASSED [ 5%]
  100. 218.57 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_unmatched_tool_calls[call_1-None] PASSED [ 5%]
  101. 220.25 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_unmatched_tool_calls[call_1-tool_call_results1] PASSED [ 5%]
  102. 222.06 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_unmatched_tool_calls[None-tool_call_results2] PASSED [ 5%]
  103. 223.69 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_non_native_tool_history PASSED [ 5%]
  104. 225.31 s tests/adapters/test_chat_adapter.py::test_non_native_tool_history_remains_text_based[adapter0] PASSED [ 5%]
  105. 227.77 s tests/adapters/test_chat_adapter.py::test_non_native_tool_history_remains_text_based[adapter1] PASSED [ 5%]
  106. 230.37 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_accepts_custom_history_formatter_returning_messages_only PASSED [ 5%]
  107. 232.84 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_tool_input PASSED [ 6%]
  108. 235.26 s tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_kitchen_sink PASSED [ 6%]
  109. 237.59 s tests/adapters/test_chat_adapter.py::test_chat_adapter_with_pydantic_models PASSED [ 6%]
  110. 240.13 s tests/adapters/test_chat_adapter.py::test_chat_adapter_signature_information PASSED [ 6%]
  111. 242.20 s tests/adapters/test_chat_adapter.py::test_chat_adapter_exception_raised_on_failure PASSED [ 6%]
  112. 244.76 s tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image PASSED [ 6%]
  113. 247.19 s tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image_with_few_shot_examples PASSED [ 6%]
  114. 249.06 s tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image_with_nested_images PASSED [ 6%]
  115. 251.58 s tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image_with_few_shot_examples_with_nested_images PASSED [ 6%]
  116. 253.49 s tests/adapters/test_chat_adapter.py::test_chat_adapter_with_tool PASSED [ 6%]
  117. 254.92 s tests/adapters/test_chat_adapter.py::test_chat_adapter_with_code PASSED [ 6%]
  118. 256.54 s tests/adapters/test_chat_adapter.py::test_code_output_field_omits_json_schema_in_prompt PASSED [ 6%]
  119. 258.79 s tests/adapters/test_chat_adapter.py::test_citations_output_field_keeps_json_schema_in_prompt PASSED [ 6%]
  120. 260.69 s tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_conversation_history PASSED [ 7%]
  121. 262.13 s tests/adapters/test_chat_adapter.py::test_chat_adapter_fallback_to_json_adapter_on_exception PASSED [ 7%]
  122. 263.74 s tests/adapters/test_chat_adapter.py::test_chat_adapter_fallback_preserves_native_function_calling_flag PASSED [ 7%]
  123. 265.74 s tests/adapters/test_chat_adapter.py::test_chat_adapter_respects_use_json_adapter_fallback_flag PASSED [ 7%]
  124. 267.95 s tests/adapters/test_chat_adapter.py::test_chat_adapter_fallback_to_json_adapter_on_exception_async PASSED [ 7%]
  125. 269.36 s tests/adapters/test_chat_adapter.py::test_chat_adapter_async_fallback_preserves_native_function_calling_flag PASSED [ 7%]
  126. 270.83 s tests/adapters/test_chat_adapter.py::test_chat_adapter_toolcalls_native_function_calling PASSED [ 7%]
  127. 272.80 s tests/adapters/test_chat_adapter.py::test_chat_adapter_toolcalls_vague_match PASSED [ 7%]
  128. 274.89 s tests/adapters/test_chat_adapter.py::test_chat_adapter_native_reasoning PASSED [ 7%]
  129. 276.31 s tests/adapters/test_chat_adapter.py::test_chat_adapter_parses_float_with_underscores PASSED [ 7%]
  130. 278.15 s tests/adapters/test_chat_adapter.py::test_format_system_message PASSED [ 7%]
  131. 281.05 s tests/adapters/test_chat_adapter.py::test_null_content_raises_adapter_parse_error PASSED [ 7%]
  132. 282.63 s tests/adapters/test_chat_adapter.py::test_empty_string_content_raises_adapter_parse_error PASSED [ 7%]
  133. 284.13 s tests/adapters/test_chat_adapter.py::test_tool_call_with_null_content_does_not_raise PASSED [ 8%]
  134. 286.23 s tests/adapters/test_chat_adapter.py::test_tool_call_with_unstructured_content_does_not_raise PASSED [ 8%]
  135. 289.01 s tests/adapters/test_chat_adapter.py::test_tool_call_with_structured_content_preserves_other_outputs PASSED [ 8%]
  136. 290.47 s tests/adapters/test_chat_adapter.py::test_provider_tool_calls_preserve_id_and_repair_arguments PASSED [ 8%]
  137. 291.90 s tests/adapters/test_citation.py::test_citation_validate_input PASSED [ 8%]
  138. 293.72 s tests/adapters/test_citation.py::test_citations_in_nested_type PASSED [ 8%]
  139. 296.32 s tests/adapters/test_citation.py::test_citation_with_all_fields PASSED [ 8%]
  140. 298.30 s tests/adapters/test_citation.py::test_citation_format PASSED [ 8%]
  141. 299.84 s tests/adapters/test_citation.py::test_citations_format PASSED [ 8%]
  142. 302.19 s tests/adapters/test_citation.py::test_citations_from_dict_list PASSED [ 8%]
  143. 304.77 s tests/adapters/test_citation.py::test_citations_postprocessing PASSED [ 8%]
  144. 306.81 s tests/adapters/test_citation.py::test_citation_extraction_from_lm_response PASSED [ 8%]
  145. 309.59 s tests/adapters/test_code.py::test_code_validate_input PASSED [ 8%]
  146. 311.77 s tests/adapters/test_code.py::test_code_in_nested_type PASSED [ 9%]
  147. 313.80 s tests/adapters/test_code.py::test_code_with_language PASSED [ 9%]
  148. 316.84 s tests/adapters/test_code.py::test_code_parses_from_dirty_code PASSED [ 9%]
  149. 318.71 s tests/adapters/test_document.py::test_document_validate_input PASSED [ 9%]
  150. 320.73 s tests/adapters/test_document.py::test_document_in_nested_type PASSED [ 9%]
  151. 323.53 s tests/adapters/test_document.py::test_document_with_all_fields PASSED [ 9%]
  152. 325.08 s tests/adapters/test_document.py::test_document_format PASSED [ 9%]
  153. 326.55 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_for_simple_signature PASSED [ 9%]
  154. 328.46 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_demo_and_typed_output PASSED [ 9%]
  155. 330.48 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_described_and_bool_outputs PASSED [ 9%]
  156. 331.84 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_history_demo_pydantic_tools_and_image PASSED [ 9%]
  157. 333.30 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_int_and_mapping_outputs PASSED [ 9%]
  158. 335.53 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_literal_and_enum_outputs PASSED [ 9%]
  159. 337.29 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_nested_pydantic_output PASSED [ 9%]
  160. 338.68 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_incomplete_demo PASSED [ 10%]
  161. 340.32 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_and_lm_kwargs_with_native_tool_calling PASSED [ 10%]
  162. 342.33 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_tool_calls_output_demo PASSED [ 10%]
  163. 344.06 s tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_non_native_tool_result_history_field PASSED [ 10%]
  164. 345.49 s tests/adapters/test_json_adapter.py::test_json_adapter_passes_structured_output_when_supported_by_model PASSED [ 10%]
  165. 347.44 s tests/adapters/test_json_adapter.py::test_json_adapter_not_using_structured_outputs_when_not_supported_by_model PASSED [ 10%]
  166. 349.64 s tests/adapters/test_json_adapter.py::test_json_adapter_with_structured_outputs_does_not_mutate_original_signature PASSED [ 10%]
  167. 351.03 s tests/adapters/test_json_adapter.py::test_json_adapter_sync_call PASSED [ 10%]
  168. 352.59 s tests/adapters/test_json_adapter.py::test_json_adapter_async_call PASSED [ 10%]
  169. 354.48 s tests/adapters/test_json_adapter.py::test_json_adapter_on_pydantic_model PASSED [ 10%]
  170. 356.87 s tests/adapters/test_json_adapter.py::test_json_adapter_parse_raise_error_on_mismatch_fields PASSED [ 10%]
  171. 358.33 s tests/adapters/test_json_adapter.py::test_json_adapter_formats_image PASSED [ 10%]
  172. 360.01 s tests/adapters/test_json_adapter.py::test_json_adapter_formats_image_with_few_shot_examples PASSED [ 10%]
  173. 361.76 s tests/adapters/test_json_adapter.py::test_json_adapter_formats_image_with_nested_images PASSED [ 11%]
  174. 363.54 s tests/adapters/test_json_adapter.py::test_json_adapter_formats_with_nested_documents PASSED [ 11%]
  175. 365.36 s tests/adapters/test_json_adapter.py::test_json_adapter_formats_image_with_few_shot_examples_with_nested_images PASSED [ 11%]
  176. 367.10 s tests/adapters/test_json_adapter.py::test_json_adapter_with_tool PASSED [ 11%]
  177. 368.89 s tests/adapters/test_json_adapter.py::test_json_adapter_with_code PASSED [ 11%]
  178. 370.53 s tests/adapters/test_json_adapter.py::test_json_adapter_formats_conversation_history PASSED [ 11%]
  179. 372.56 s tests/adapters/test_json_adapter.py::test_json_adapter_on_pydantic_model_async PASSED [ 11%]
  180. 374.35 s tests/adapters/test_json_adapter.py::test_json_adapter_does_not_fallback_to_json_mode_on_structured_output_lm_error PASSED [ 11%]
  181. 375.96 s tests/adapters/test_json_adapter.py::test_json_adapter_json_mode_no_structured_outputs PASSED [ 11%]
  182. 377.63 s tests/adapters/test_json_adapter.py::test_json_adapter_json_mode_no_structured_outputs_async PASSED [ 11%]
  183. 379.35 s tests/adapters/test_json_adapter.py::test_json_adapter_does_not_fallback_to_json_mode_on_structured_output_lm_error_async PASSED [ 11%]
  184. 381.35 s tests/adapters/test_json_adapter.py::test_error_message_on_json_adapter_failure PASSED [ 11%]
  185. 383.33 s tests/adapters/test_json_adapter.py::test_error_message_on_json_adapter_failure_async PASSED [ 11%]
  186. 385.23 s tests/adapters/test_json_adapter.py::test_json_adapter_toolcalls_native_function_calling PASSED [ 12%]
  187. 387.20 s tests/adapters/test_json_adapter.py::test_json_adapter_toolcalls_no_native_function_calling PASSED [ 12%]
  188. 389.20 s tests/adapters/test_json_adapter.py::test_json_adapter_native_reasoning PASSED [ 12%]
  189. 391.33 s tests/adapters/test_json_adapter.py::test_json_adapter_with_responses_api PASSED [ 12%]
  190. 393.16 s tests/adapters/test_json_adapter.py::test_format_system_message PASSED [ 12%]
  191. 395.17 s tests/adapters/test_reasoning.py::test_reasoning_basic_operations PASSED [ 12%]
  192. 397.49 s tests/adapters/test_reasoning.py::test_reasoning_concatenation PASSED [ 12%]
  193. 399.53 s tests/adapters/test_reasoning.py::test_reasoning_string_methods PASSED [ 12%]
  194. 401.57 s tests/adapters/test_reasoning.py::test_reasoning_with_chain_of_thought PASSED [ 12%]
  195. 403.74 s tests/adapters/test_reasoning.py::test_reasoning_error_message PASSED [ 12%]
  196. 405.75 s tests/adapters/test_tool.py::test_basic_initialization PASSED [ 12%]
  197. 407.44 s tests/adapters/test_tool.py::test_tool_from_function PASSED [ 12%]
  198. 409.12 s tests/adapters/test_tool.py::test_tool_from_class PASSED [ 12%]
  199. 411.16 s tests/adapters/test_tool.py::test_tool_from_function_with_pydantic PASSED [ 13%]
  200. 412.96 s tests/adapters/test_tool.py::test_tool_from_function_with_pydantic_nesting PASSED [ 13%]
  201. 414.74 s tests/adapters/test_tool.py::test_tool_callable PASSED [ 13%]
  202. 416.69 s tests/adapters/test_tool.py::test_tool_with_pydantic_callable PASSED [ 13%]
  203. 418.77 s tests/adapters/test_tool.py::test_invalid_function_call PASSED [ 13%]
  204. 421.05 s tests/adapters/test_tool.py::test_parameter_desc PASSED [ 13%]
  205. 423.97 s tests/adapters/test_tool.py::test_tool_with_default_args_without_type_hints PASSED [ 13%]
  206. 426.95 s tests/adapters/test_tool.py::test_tool_call_parses_args PASSED [ 13%]
  207. 430.06 s tests/adapters/test_tool.py::test_tool_call_parses_nested_list_of_pydantic_model PASSED [ 13%]
  208. 433.07 s tests/adapters/test_tool.py::test_tool_call_kwarg PASSED [ 13%]
  209. 436.19 s tests/adapters/test_tool.py::test_tool_str PASSED [ 13%]
  210. 438.69 s tests/adapters/test_tool.py::test_async_tool_from_function PASSED [ 13%]
  211. 441.28 s tests/adapters/test_tool.py::test_async_tool_with_pydantic PASSED [ 13%]
  212. 444.24 s tests/adapters/test_tool.py::test_async_tool_with_complex_pydantic PASSED [ 13%]
  213. 446.00 s tests/adapters/test_tool.py::test_async_tool_invalid_call PASSED [ 14%]
  214. 447.57 s tests/adapters/test_tool.py::test_async_tool_with_kwargs PASSED [ 14%]
  215. 449.79 s tests/adapters/test_tool.py::test_async_concurrent_calls PASSED [ 14%]
  216. 452.25 s tests/adapters/test_tool.py::test_async_tool_call_in_sync_mode PASSED [ 14%]
  217. 454.30 s tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data0-expected0] PASSED [ 14%]
  218. 456.64 s tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data1-expected1] PASSED [ 14%]
  219. 459.09 s tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data2-expected2] PASSED [ 14%]
  220. 461.81 s tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data3-expected3] PASSED [ 14%]
  221. 464.07 s tests/adapters/test_tool.py::test_tool_calls_format_from_dict_list PASSED [ 14%]
  222. 466.53 s tests/adapters/test_tool.py::test_tool_calls_preserves_ids_from_dict_list_and_format_omits_ids PASSED [ 14%]
  223. 469.00 s tests/adapters/test_tool.py::test_tool_calls_json_schema_omits_internal_id_field PASSED [ 14%]
  224. 470.64 s tests/adapters/test_tool.py::test_tool_calls_can_carry_results_without_formatting_them_for_lm PASSED [ 14%]
  225. 472.12 s tests/adapters/test_tool.py::test_tool_call_results_from_tool_calls_and_values PASSED [ 14%]
  226. 474.01 s tests/adapters/test_tool.py::test_tool_call_results_history_serialization_round_trip PASSED [ 15%]
  227. 476.13 s tests/adapters/test_tool.py::test_tool_call_results_can_round_trip_as_native_tool_result_message PASSED [ 15%]
  228. 478.05 s tests/adapters/test_tool.py::test_toolcalls_vague_match PASSED [ 15%]
  229. 480.22 s tests/adapters/test_tool.py::test_tool_convert_input_schema_to_tool_args_no_input_params PASSED [ 15%]
  230. 482.38 s tests/adapters/test_tool.py::test_tool_convert_input_schema_to_tool_args_lang_chain PASSED [ 15%]
  231. 484.76 s tests/adapters/test_tool.py::test_tool_call_execute PASSED [ 15%]
  232. 486.98 s tests/adapters/test_tool.py::test_tool_call_execute_with_local_functions PASSED [ 15%]
  233. 489.56 s tests/adapters/test_two_step_adapter.py::test_two_step_adapter_format_exact_messages_for_simple_signature_with_demo PASSED [ 15%]
  234. 492.65 s tests/adapters/test_two_step_adapter.py::test_two_step_adapter_format_exact_messages_with_typed_outputs PASSED [ 15%]
  235. 495.70 s tests/adapters/test_two_step_adapter.py::test_two_step_adapter_call PASSED [ 15%]
  236. 499.09 s tests/adapters/test_two_step_adapter.py::test_two_step_adapter_async_call PASSED [ 15%]
  237. 502.33 s tests/adapters/test_two_step_adapter.py::test_two_step_adapter_parse PASSED [ 15%]
  238. 504.88 s tests/adapters/test_two_step_adapter.py::test_two_step_adapter_parse_errors PASSED [ 15%]
  239. 507.76 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_and_parse_basic PASSED [ 16%]
  240. 510.97 s tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_multiple_fields PASSED [ 16%]
  241. 513.73 s tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_raises_on_missing_field PASSED [ 16%]
  242. 516.13 s tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_casts_types PASSED [ 16%]
  243. 518.52 s tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_raises_on_type_error PASSED [ 16%]
  244. 521.28 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_and_parse_nested_model PASSED [ 16%]
  245. 523.49 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_and_parse_list_of_models PASSED [ 16%]
  246. 525.60 s tests/adapters/test_xml_adapter.py::test_xml_adapter_with_tool_like_output PASSED [ 16%]
  247. 528.05 s tests/adapters/test_xml_adapter.py::test_xml_adapter_formats_nested_images PASSED [ 16%]
  248. 530.37 s tests/adapters/test_xml_adapter.py::test_xml_adapter_with_code PASSED [ 16%]
  249. 532.85 s tests/adapters/test_xml_adapter.py::test_xml_adapter_full_prompt PASSED [ 16%]
  250. 534.75 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_for_simple_signature PASSED [ 16%]
  251. 536.96 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_non_native_tool_result_history_field PASSED [ 16%]
  252. 539.21 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_for_two_input_signature PASSED [ 17%]
  253. 541.59 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_demo_and_typed_output PASSED [ 17%]
  254. 543.99 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_history_demo_pydantic_tools_and_image PASSED [ 17%]
  255. 546.30 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_nested_pydantic_output PASSED [ 17%]
  256. 548.58 s tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_incomplete_demo PASSED [ 17%]
  257. 550.39 s tests/adapters/test_xml_adapter.py::test_format_system_message PASSED [ 17%]
  258. 552.44 s tests/callback/test_callback.py::test_callback_injection[args0-kwargs0] PASSED [ 17%]
  259. 554.80 s tests/callback/test_callback.py::test_callback_injection[args1-kwargs1] PASSED [ 17%]
  260. 557.32 s tests/callback/test_callback.py::test_callback_injection[args2-kwargs2] PASSED [ 17%]
  261. 560.44 s tests/callback/test_callback.py::test_callback_injection[args3-kwargs3] PASSED [ 17%]
  262. 563.12 s tests/callback/test_callback.py::test_callback_injection_local PASSED [ 17%]
  263. 566.51 s tests/callback/test_callback.py::test_callback_error_handling PASSED [ 17%]
  264. 569.29 s tests/callback/test_callback.py::test_multiple_callbacks PASSED [ 17%]
  265. 572.70 s tests/callback/test_callback.py::test_callback_complex_module PASSED [ 18%]
  266. 576.90 s tests/callback/test_callback.py::test_callback_async_module PASSED [ 18%]
  267. 579.92 s tests/callback/test_callback.py::test_tool_calls PASSED [ 18%]
  268. 582.19 s tests/callback/test_callback.py::test_active_id PASSED [ 18%]
  269. 586.02 s tests/clients/test_cache.py::test_initialization PASSED [ 18%]
  270. 587.75 s tests/clients/test_cache.py::test_invalid_cache_initialization PASSED [ 18%]
  271. 591.90 s tests/clients/test_cache.py::test_cache_key_generation PASSED [ 18%]
  272. 595.76 s tests/clients/test_cache.py::test_put_and_get PASSED [ 18%]
  273. 599.89 s tests/clients/test_cache.py::test_cache_miss PASSED [ 18%]
  274. 602.83 s tests/clients/test_cache.py::test_unserializable_key PASSED [ 18%]
  275. 605.16 s tests/clients/test_cache.py::test_reset_memory_cache PASSED [ 18%]
  276. 608.25 s tests/clients/test_cache.py::test_save_and_load_memory_cache PASSED [ 18%]
  277. 610.91 s tests/clients/test_cache.py::test_request_cache_decorator PASSED [ 18%]
  278. 614.90 s tests/clients/test_cache.py::test_request_cache_decorator_with_ignored_args_for_cache_key PASSED [ 18%]
  279. 618.88 s tests/clients/test_cache.py::test_request_cache_decorator_async PASSED [ 19%]
  280. 622.82 s tests/clients/test_cache.py::test_cache_consistency_with_lm_call_modifies_the_request PASSED [ 19%]
  281. 624.92 s tests/clients/test_cache.py::test_cache_fallback_on_restricted_environment PASSED [ 19%]
  282. 626.77 s tests/clients/test_cache.py::test_cache_init_with_disk_disabled_and_none_dir PASSED [ 19%]
  283. 630.69 s tests/clients/test_cache.py::test_model_response_roundtrip_in_restricted_mode PASSED [ 19%]
  284. 634.49 s tests/clients/test_cache.py::test_registered_dataclass_roundtrip_in_restricted_mode PASSED [ 19%]
  285. 638.10 s tests/clients/test_cache.py::test_configure_cache_registers_safe_types PASSED [ 19%]
  286. 642.78 s tests/clients/test_cache.py::test_corrupt_disk_entries_return_none PASSED [ 19%]
  287. 647.78 s tests/clients/test_cache.py::test_restricted_and_unrestricted_share_wire_format
  288. 649.02 s PASSED [ 19%]
  289. 654.03 s tests/clients/test_cache.py::test_unlisted_type_rejected_on_read
  290. 654.60 s PASSED [ 19%]
  291. 657.37 s tests/clients/test_cache.py::test_safe_types_rejects_non_types PASSED [ 19%]
  292. 657.38 s tests/clients/test_databricks.py::test_create_directory_in_databricks_unity_catalog SKIPPED [ 19%]
  293. 657.38 s tests/clients/test_databricks.py::test_create_finetuning_job SKIPPED [ 19%]
  294. 657.38 s tests/clients/test_databricks.py::test_deploy_finetuned_model SKIPPED [ 20%]
  295. 660.81 s tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[dict] PASSED [ 20%]
  296. 663.95 s tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[list] PASSED [ 20%]
  297. 666.26 s tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[str] PASSED [ 20%]
  298. 668.56 s tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[int] PASSED [ 20%]
  299. 671.46 s tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[tuple] PASSED [ 20%]
  300. 673.69 s tests/clients/test_disk_serialization.py::test_pydantic_roundtrip PASSED [ 20%]
  301. 675.34 s tests/clients/test_disk_serialization.py::test_dataclass_roundtrip PASSED [ 20%]
  302. 677.17 s tests/clients/test_disk_serialization.py::test_ndarray_roundtrip PASSED [ 20%]
  303. 679.39 s tests/clients/test_disk_serialization.py::test_litellm_response_roundtrip PASSED [ 20%]
  304. 681.33 s tests/clients/test_disk_serialization.py::test_text_completion_response_roundtrip PASSED [ 20%]
  305. 683.62 s tests/clients/test_disk_serialization.py::test_embedding_response_roundtrip PASSED [ 20%]
  306. 686.75 s tests/clients/test_disk_serialization.py::test_responses_api_roundtrip PASSED [ 20%]
  307. 690.19 s tests/clients/test_disk_serialization.py::test_tool_call_response_roundtrip PASSED [ 21%]
  308. 695.22 s tests/clients/test_disk_serialization.py::test_cached_tool_call_normalizes_after_fresh_process_roundtrip
  309. 698.52 s PASSED [ 21%]
  310. 703.53 s tests/clients/test_disk_serialization.py::test_cached_responses_function_call_normalizes_after_fresh_process_roundtrip
  311. 705.53 s PASSED [ 21%]
  312. 708.55 s tests/clients/test_disk_serialization.py::test_unlisted_type_blocked_on_read PASSED [ 21%]
  313. 711.80 s tests/clients/test_disk_serialization.py::test_allowlists_are_isolated PASSED [ 21%]
  314. 714.12 s tests/clients/test_disk_serialization.py::test_numpy_ctypeslib_blocked PASSED [ 21%]
  315. 716.43 s tests/clients/test_disk_serialization.py::test_restricted_load_rejects_unknown_type PASSED [ 21%]
  316. 718.98 s tests/clients/test_disk_serialization.py::test_corrupt_pickle_raises_deserialization_error PASSED [ 21%]
  317. 721.02 s tests/clients/test_disk_serialization.py::test_nested_class_safe_types PASSED [ 21%]
  318. 723.19 s tests/clients/test_disk_serialization.py::test_restricted_load_allows_builtin_types PASSED [ 21%]
  319. 725.35 s tests/clients/test_disk_serialization.py::test_all_cached_lm_types_roundtrip PASSED [ 21%]
  320. 727.78 s tests/clients/test_embedding.py::test_litellm_embedding PASSED [ 21%]
  321. 730.02 s tests/clients/test_embedding.py::test_callable_embedding PASSED [ 21%]
  322. 734.19 s tests/clients/test_embedding.py::test_callable_numpy_embedding_persists_to_disk PASSED [ 22%]
  323. 735.70 s tests/clients/test_embedding.py::test_invalid_model_type PASSED [ 22%]
  324. 737.65 s tests/clients/test_embedding.py::test_async_embedding PASSED [ 22%]
  325. 740.25 s tests/clients/test_embedding.py::test_call_caching_false_overrides_instance_true PASSED [ 22%]
  326. 743.03 s tests/clients/test_embedding.py::test_call_caching_true_overrides_instance_false PASSED [ 22%]
  327. 745.27 s tests/clients/test_embedding.py::test_acall_caching_false_overrides_instance_true PASSED [ 22%]
  328. 747.52 s tests/clients/test_embedding.py::test_acall_caching_true_overrides_instance_false PASSED [ 22%]
  329. 749.69 s tests/clients/test_inspect_global_history.py::test_inspect_history_basic PASSED [ 22%]
  330. 751.78 s tests/clients/test_inspect_global_history.py::test_inspect_history_renders_message_tool_calls PASSED [ 22%]
  331. 754.10 s tests/clients/test_inspect_global_history.py::test_inspect_history_renders_output_tool_calls_without_text PASSED [ 22%]
  332. 757.00 s tests/clients/test_inspect_global_history.py::test_inspect_history_with_n PASSED [ 22%]
  333. 759.12 s tests/clients/test_inspect_global_history.py::test_inspect_empty_history PASSED [ 22%]
  334. 761.25 s tests/clients/test_inspect_global_history.py::test_inspect_history_n_larger_than_history PASSED [ 22%]
  335. 763.35 s tests/clients/test_lazy_litellm_import.py::test_import_dspy_does_not_import_litellm PASSED [ 22%]
  336. 765.51 s tests/clients/test_lazy_litellm_import.py::test_lm_litellm_use_raises_helpful_error_without_litellm PASSED [ 23%]
  337. 767.66 s tests/clients/test_lazy_litellm_import.py::test_embedder_litellm_use_raises_helpful_error_without_litellm PASSED [ 23%]
  338. 769.77 s tests/clients/test_lazy_litellm_import.py::test_concurrent_lm_first_use_materializes_litellm_once PASSED [ 23%]
  339. 774.79 s tests/clients/test_lm.py::test_chat_lms_can_be_queried
  340. 775.24 s PASSED [ 23%]
  341. 780.24 s tests/clients/test_lm.py::test_dspy_cache
  342. 782.83 s PASSED [ 23%]
  343. 785.15 s tests/clients/test_lm.py::test_disabled_cache_skips_cache_key PASSED [ 23%]
  344. 788.68 s tests/clients/test_lm.py::test_rollout_id_bypasses_cache PASSED [ 23%]
  345. 790.53 s tests/clients/test_lm.py::test_zero_temperature_rollout_warns_once PASSED [ 23%]
  346. 792.72 s tests/clients/test_lm.py::test_rollout_id_with_default_temperature_does_not_warn PASSED [ 23%]
  347. 797.72 s tests/clients/test_lm.py::test_text_lms_can_be_queried
  348. 798.13 s PASSED [ 23%]
  349. 803.13 s tests/clients/test_lm.py::test_lm_calls_support_callables
  350. 803.66 s PASSED [ 23%]
  351. 808.66 s tests/clients/test_lm.py::test_lm_calls_support_pydantic_models
  352. 808.78 s PASSED [ 23%]
  353. 810.65 s tests/clients/test_lm.py::test_lm_wraps_litellm_errors_with_metadata PASSED [ 23%]
  354. 812.36 s tests/clients/test_lm.py::test_lm_wraps_litellm_context_window_error PASSED [ 24%]
  355. 814.28 s tests/clients/test_lm.py::test_lm_wraps_unknown_boundary_error_as_unexpected_error PASSED [ 24%]
  356. 816.26 s tests/clients/test_lm.py::test_lm_preserves_existing_lm_error_without_self_cause PASSED [ 24%]
  357. 817.99 s tests/clients/test_lm.py::test_lm_preserves_existing_lm_error_without_self_cause_async PASSED [ 24%]
  358. 819.77 s tests/clients/test_lm.py::test_retry_number_set_correctly PASSED [ 24%]
  359. 824.62 s tests/clients/test_lm.py::test_retry_made_on_system_errors PASSED [ 24%]
  360. 826.69 s tests/clients/test_lm.py::test_reasoning_model_token_parameter PASSED [ 24%]
  361. 828.48 s tests/clients/test_lm.py::test_reasoning_model_requirements[openai/o1] PASSED [ 24%]
  362. 830.72 s tests/clients/test_lm.py::test_reasoning_model_requirements[openai/gpt-5-nano] PASSED [ 24%]
  363. 832.61 s tests/clients/test_lm.py::test_reasoning_model_requirements[openai/gpt-5-mini] PASSED [ 24%]
  364. 834.31 s tests/clients/test_lm.py::test_gpt_5_chat_not_reasoning_model PASSED [ 24%]
  365. 836.70 s tests/clients/test_lm.py::test_base_lm_init_uses_lm_defaults_and_isolates_callback_list PASSED [ 24%]
  366. 839.23 s tests/clients/test_lm.py::test_base_lm_forward_contract_defaults_to_legacy PASSED [ 24%]
  367. 841.46 s tests/clients/test_lm.py::test_base_lm_forward_contract_accepts_explicit_values PASSED [ 25%]
  368. 844.14 s tests/clients/test_lm.py::test_base_lm_forward_contract_rejects_unknown_values PASSED [ 25%]
  369. 846.52 s tests/clients/test_lm.py::test_base_lm_validates_typed_lm_response PASSED [ 25%]
  370. 849.01 s tests/clients/test_lm.py::test_base_lm_warns_when_inherited_legacy_forward_returns_lm_response PASSED [ 25%]
  371. 851.22 s tests/clients/test_lm.py::test_base_lm_errors_when_explicit_legacy_forward_returns_lm_response PASSED [ 25%]
  372. 854.02 s tests/clients/test_lm.py::test_base_lm_inherited_legacy_forward_returning_lm_response_errors_on_direct_call PASSED [ 25%]
  373. 856.80 s tests/clients/test_lm.py::test_base_lm_default_call_keeps_legacy_outputs PASSED [ 25%]
  374. 859.40 s tests/clients/test_lm.py::test_base_lm_experimental_call_returns_lm_response_through_legacy_bridge PASSED [ 25%]
  375. 862.02 s tests/clients/test_lm.py::test_base_lm_explicit_lm_request_returns_lm_response_without_experimental PASSED [ 25%]
  376. 864.13 s tests/clients/test_lm.py::test_base_lm_legacy_bridge_records_typed_history_and_usage_once PASSED [ 25%]
  377. 866.35 s tests/clients/test_lm.py::test_base_lm_typed_forward_contract_uses_lm_request PASSED [ 25%]
  378. 868.07 s tests/clients/test_lm.py::test_base_lm_typed_forward_contract_rejects_non_lm_response_at_call_time PASSED [ 25%]
  379. 869.78 s tests/clients/test_lm.py::test_base_lm_request_call_rejects_mixed_inputs PASSED [ 25%]
  380. 871.99 s tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_system_user_and_assistant_turns[current_lm] PASSED [ 26%]
  381. 874.51 s tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_system_user_and_assistant_turns[typed_lm] PASSED [ 26%]
  382. 877.06 s tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_tool_call_transcripts[current_lm] PASSED [ 26%]
  383. 879.67 s tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_tool_call_transcripts[typed_lm] PASSED [ 26%]
  384. 882.04 s tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_can_reuse_lm_response_as_assistant_turn[current_lm] PASSED [ 26%]
  385. 884.10 s tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_can_reuse_lm_response_as_assistant_turn[typed_lm] PASSED [ 26%]
  386. 886.09 s tests/clients/test_lm.py::test_base_lm_async_explicit_lm_request_returns_lm_response PASSED [ 26%]
  387. 888.03 s tests/clients/test_lm.py::test_base_lm_tracks_usage_for_custom_subclasses PASSED [ 26%]
  388. 889.51 s tests/clients/test_lm.py::test_base_lm_copy_is_shallow_runtime_copy_with_isolated_dspy_state PASSED [ 26%]
  389. 891.13 s tests/clients/test_lm.py::test_dump_state PASSED [ 26%]
  390. 892.90 s tests/clients/test_lm.py::test_reasoning_model_dump_state_uses_constructor_max_tokens PASSED [ 26%]
  391. 894.76 s tests/clients/test_lm.py::test_dump_state_preserves_enabled_developer_role PASSED [ 26%]
  392. 897.02 s tests/clients/test_lm.py::test_dump_state_ignores_internal_class_marker_kwarg PASSED [ 26%]
  393. 899.04 s tests/clients/test_lm.py::test_load_state PASSED [ 27%]
  394. 900.85 s tests/clients/test_lm.py::test_reasoning_model_load_state_round_trips_canonical_state PASSED [ 27%]
  395. 903.32 s tests/clients/test_lm.py::test_reasoning_model_load_state_accepts_max_completion_tokens_alias PASSED [ 27%]
  396. 904.97 s tests/clients/test_lm.py::test_lm_load_state_forwards_allow_custom_lm_class PASSED [ 27%]
  397. 909.97 s tests/clients/test_lm.py::test_exponential_backoff_retry
  398. 910.01 s PASSED [ 27%]
  399. 912.36 s tests/clients/test_lm.py::test_logprobs_included_when_requested PASSED [ 27%]
  400. 914.44 s tests/clients/test_lm.py::test_async_lm_call PASSED [ 27%]
  401. 919.44 s tests/clients/test_lm.py::test_async_lm_call_with_cache
  402. 920.94 s PASSED [ 27%]
  403. 923.74 s tests/clients/test_lm.py::test_lm_history_size_limit PASSED [ 27%]
  404. 926.52 s tests/clients/test_lm.py::test_disable_history PASSED [ 27%]
  405. 928.65 s tests/clients/test_lm.py::test_responses_api PASSED [ 27%]
  406. 931.44 s tests/clients/test_lm.py::test_lm_replaces_system_with_developer_role PASSED [ 27%]
  407. 935.90 s tests/clients/test_lm.py::test_responses_api_tool_calls PASSED [ 27%]
  408. 938.41 s tests/clients/test_lm.py::test_reasoning_effort_responses_api PASSED [ 27%]
  409. 940.37 s tests/clients/test_lm.py::test_call_reasoning_model_with_chat_api PASSED [ 28%]
  410. 943.02 s tests/clients/test_lm.py::test_api_key_not_saved_in_json PASSED [ 28%]
  411. 945.08 s tests/clients/test_lm.py::test_responses_api_converts_images_correctly PASSED [ 28%]
  412. 947.29 s tests/clients/test_lm.py::test_responses_api_converts_files_correctly PASSED [ 28%]
  413. 949.65 s tests/clients/test_lm.py::test_responses_api_preserves_multi_message_structure PASSED [ 28%]
  414. 952.20 s tests/clients/test_lm.py::test_responses_api_with_image_input PASSED [ 28%]
  415. 954.75 s tests/clients/test_lm.py::test_responses_api_with_pydantic_model_input PASSED [ 28%]
  416. 957.37 s tests/clients/test_lm.py::test_responses_api_with_none_usage PASSED [ 28%]
  417. 959.49 s tests/clients/test_lm.py::test_responses_api_with_none_usage_async PASSED [ 28%]
  418. 961.48 s tests/clients/test_lm.py::test_streaming_passes_headers_correctly PASSED [ 28%]
  419. 961.61 s tests/clients/test_lm_direct_live.py::test_live_openai_chat_direct_system_user_assistant_multiturn SKIPPED [ 28%]
  420. 961.61 s tests/clients/test_lm_direct_live.py::test_live_openai_chat_direct_tool_call_transcript SKIPPED [ 28%]
  421. 961.61 s tests/clients/test_lm_direct_live.py::test_live_openai_chat_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 28%]
  422. 961.61 s tests/clients/test_lm_direct_live.py::test_live_openai_responses_direct_system_user_assistant_multiturn SKIPPED [ 29%]
  423. 961.61 s tests/clients/test_lm_direct_live.py::test_live_openai_responses_direct_tool_call_transcript SKIPPED [ 29%]
  424. 961.61 s tests/clients/test_lm_direct_live.py::test_live_openai_responses_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 29%]
  425. 961.61 s tests/clients/test_lm_direct_live.py::test_live_anthropic_chat_direct_system_user_assistant_multiturn SKIPPED [ 29%]
  426. 961.61 s tests/clients/test_lm_direct_live.py::test_live_anthropic_chat_direct_tool_call_transcript SKIPPED [ 29%]
  427. 961.61 s tests/clients/test_lm_direct_live.py::test_live_anthropic_chat_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 29%]
  428. 961.61 s tests/clients/test_lm_direct_live.py::test_live_gemini_chat_direct_system_user_assistant_multiturn SKIPPED [ 29%]
  429. 961.61 s tests/clients/test_lm_direct_live.py::test_live_gemini_chat_direct_tool_call_transcript SKIPPED [ 29%]
  430. 961.61 s tests/clients/test_lm_direct_live.py::test_live_gemini_chat_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 29%]
  431. 963.66 s tests/clients/test_lm_local.py::test_command_with_spaces_in_path PASSED [ 29%]
  432. 966.29 s tests/clients/test_lm_local.py::test_command_construction_prevents_injection PASSED [ 29%]
  433. 968.49 s tests/clients/test_lm_local.py::test_command_is_list_not_string PASSED [ 29%]
  434. 970.36 s tests/core/test_types.py::test_message_content_and_tool_calls_normalize_for_dspy_history_surface PASSED [ 29%]
  435. 972.51 s tests/core/test_types.py::test_tool_result_content_none_normalizes_to_empty_parts PASSED [ 30%]
  436. 974.73 s tests/core/test_types.py::test_audio_content_accepts_url_and_history_preserves_url PASSED [ 30%]
  437. 977.03 s tests/core/test_types.py::test_audio_content_defaults_null_format_to_wav PASSED [ 30%]
  438. 979.25 s tests/core/test_types.py::test_image_content_requires_mapping_with_url PASSED [ 30%]
  439. 981.42 s tests/core/test_types.py::test_video_data_round_trips_through_history_messages PASSED [ 30%]
  440. 983.41 s tests/core/test_types.py::test_document_source_url_stays_url_and_round_trips_through_history_messages PASSED [ 30%]
  441. 985.61 s tests/core/test_types.py::test_config_extensions_flatten_when_converted_to_legacy_kwargs PASSED [ 30%]
  442. 988.03 s tests/core/test_types.py::test_lm_kwargs_aliases_normalize_for_existing_dspy_lm_callers PASSED [ 30%]
  443. 990.37 s tests/core/test_types.py::test_nested_config_aliases_remain_supported_for_existing_interfaces PASSED [ 30%]
  444. 992.31 s tests/core/test_types.py::test_usage_normalizes_existing_user_visible_token_aliases PASSED [ 30%]
  445. 994.07 s tests/core/test_types.py::test_default_config_does_not_serialize_empty_stop_sequences PASSED [ 30%]
  446. 996.20 s tests/core/test_types.py::test_history_entry_exposes_existing_dict_style_properties PASSED [ 30%]
  447. 998.04 s tests/core/test_types.py::test_response_rejects_empty_outputs PASSED [ 30%]
  448. 1000.03 s tests/core/test_types.py::test_output_to_value_preserves_redacted_thinking_part PASSED [ 31%]
  449. 1002.02 s tests/core/test_types.py::test_stream_event_indices_must_be_non_negative PASSED [ 31%]
  450. 1003.71 s tests/core/test_types.py::test_stream_builder_rejects_sparse_output_indices PASSED [ 31%]
  451. 1005.57 s tests/core/test_types.py::test_stream_builder_rejects_sparse_part_indices PASSED [ 31%]
  452. 1007.43 s tests/core/test_types.py::test_stream_builder_rejects_delta_type_changes PASSED [ 31%]
  453. 1009.09 s tests/core/test_types.py::test_stream_builder_rejects_incomplete_tool_call_arguments PASSED [ 31%]
  454. 1009.09 s tests/datasets/test_dataset.py::test_input_keys SKIPPED (need --extra
  455. 1009.10 s option to run) [ 31%]
  456. 1010.50 s tests/docs/test_mkdocs_links.py::test_nav_files_exist PASSED [ 31%]
  457. 1012.06 s tests/evaluate/test_auto_evaluation.py::test_semantic_f1_returns_prediction_without_trace PASSED [ 31%]
  458. 1013.96 s tests/evaluate/test_auto_evaluation.py::test_semantic_f1_returns_prediction_with_trace PASSED [ 31%]
  459. 1016.34 s tests/evaluate/test_auto_evaluation.py::test_semantic_f1_score_value PASSED [ 31%]
  460. 1018.94 s tests/evaluate/test_auto_evaluation.py::test_complete_and_grounded_returns_prediction_without_trace PASSED [ 31%]
  461. 1021.52 s tests/evaluate/test_auto_evaluation.py::test_complete_and_grounded_returns_prediction_with_trace PASSED [ 31%]
  462. 1023.65 s tests/evaluate/test_auto_evaluation.py::test_semantic_f1_prediction_can_be_compared PASSED [ 31%]
  463. 1026.06 s tests/evaluate/test_evaluate.py::test_evaluate_initialization PASSED [ 32%]
  464. 1029.17 s tests/evaluate/test_evaluate.py::test_evaluate_call PASSED [ 32%]
  465. 1031.88 s tests/evaluate/test_evaluate.py::test_evaluate_single_thread_runs_in_main_thread PASSED [ 32%]
  466. 1031.90 s tests/evaluate/test_evaluate.py::test_construct_result_df SKIPPED (need
  467. 1031.90 s --extra option to run) [ 32%]
  468. 1034.35 s tests/evaluate/test_evaluate.py::test_multithread_evaluate_call PASSED [ 32%]
  469. 1038.31 s tests/evaluate/test_evaluate.py::test_multi_thread_evaluate_call_cancelled PASSED [ 32%]
  470. 1040.14 s tests/evaluate/test_evaluate.py::test_evaluate_call_wrong_answer PASSED [ 32%]
  471. 1040.14 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example0] SKIPPED [ 32%]
  472. 1040.14 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example1] SKIPPED [ 32%]
  473. 1040.14 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example2] SKIPPED [ 32%]
  474. 1040.14 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example3] SKIPPED [ 32%]
  475. 1040.14 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example0] SKIPPED [ 32%]
  476. 1040.14 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example1] SKIPPED [ 32%]
  477. 1040.14 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example2] SKIPPED [ 33%]
  478. 1040.14 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example3] SKIPPED [ 33%]
  479. 1040.14 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example0] SKIPPED [ 33%]
  480. 1040.14 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example1] SKIPPED [ 33%]
  481. 1040.15 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example2] SKIPPED [ 33%]
  482. 1040.15 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example3] SKIPPED [ 33%]
  483. 1040.15 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example0] SKIPPED [ 33%]
  484. 1040.15 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example1] SKIPPED [ 33%]
  485. 1040.15 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example2] SKIPPED [ 33%]
  486. 1040.15 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example3] SKIPPED [ 33%]
  487. 1040.15 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example0] SKIPPED [ 33%]
  488. 1040.15 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example1] SKIPPED [ 33%]
  489. 1040.15 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example2] SKIPPED [ 33%]
  490. 1040.15 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example3] SKIPPED [ 34%]
  491. 1040.15 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example0] SKIPPED [ 34%]
  492. 1040.15 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example1] SKIPPED [ 34%]
  493. 1040.15 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example2] SKIPPED [ 34%]
  494. 1040.15 s tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example3] SKIPPED [ 34%]
  495. 1042.58 s tests/evaluate/test_evaluate.py::test_evaluate_callback PASSED [ 34%]
  496. 1044.78 s tests/evaluate/test_evaluate.py::test_evaluation_result_repr PASSED [ 34%]
  497. 1046.66 s tests/evaluate/test_evaluate.py::test_evaluate_save_as_json_with_history PASSED [ 34%]
  498. 1048.81 s tests/evaluate/test_evaluate.py::test_evaluate_save_as_csv_with_history PASSED [ 34%]
  499. 1050.82 s tests/evaluate/test_metrics.py::test_answer_exact_match_string PASSED [ 34%]
  500. 1052.78 s tests/evaluate/test_metrics.py::test_answer_exact_match_list PASSED [ 34%]
  501. 1054.94 s tests/evaluate/test_metrics.py::test_answer_exact_match_no_match PASSED [ 34%]
  502. 1056.58 s tests/metadata/test_metadata.py::test_metadata PASSED [ 34%]
  503. 1058.55 s tests/predict/test_aggregation.py::test_majority_with_prediction PASSED [ 35%]
  504. 1060.41 s tests/predict/test_aggregation.py::test_majority_with_completions PASSED [ 35%]
  505. 1062.15 s tests/predict/test_aggregation.py::test_majority_with_list PASSED [ 35%]
  506. 1063.84 s tests/predict/test_aggregation.py::test_majority_with_normalize PASSED [ 35%]
  507. 1065.52 s tests/predict/test_aggregation.py::test_majority_with_field PASSED [ 35%]
  508. 1067.19 s tests/predict/test_aggregation.py::test_majority_with_no_majority PASSED [ 35%]
  509. 1068.85 s tests/predict/test_best_of_n.py::test_refine_forward_success_first_attempt PASSED [ 35%]
  510. 1070.42 s tests/predict/test_best_of_n.py::test_refine_module_default_fail_count PASSED [ 35%]
  511. 1072.02 s tests/predict/test_best_of_n.py::test_refine_module_custom_fail_count PASSED [ 35%]
  512. 1073.56 s tests/predict/test_chain_of_thought.py::test_initialization_with_string_signature PASSED [ 35%]
  513. 1075.21 s tests/predict/test_chain_of_thought.py::test_async_chain_of_thought PASSED [ 35%]
  514. 1076.80 s tests/predict/test_chain_of_thought.py::test_chain_of_thought_with_native_reasoning PASSED [ 35%]
  515. 1078.58 s tests/predict/test_chain_of_thought.py::test_chain_of_thought_with_manual_reasoning PASSED [ 35%]
  516. 1078.60 s tests/predict/test_code_act.py::test_codeact_code_generation SKIPPED [ 36%]
  517. 1078.60 s tests/predict/test_code_act.py::test_codeact_support_multiple_fields SKIPPED [ 36%]
  518. 1078.60 s tests/predict/test_code_act.py::test_codeact_code_parse_failure SKIPPED [ 36%]
  519. 1078.60 s tests/predict/test_code_act.py::test_codeact_code_execution_failure SKIPPED [ 36%]
  520. 1078.60 s tests/predict/test_code_act.py::test_codeact_tool_validation SKIPPED [ 36%]
  521. 1078.60 s tests/predict/test_knn.py::test_knn_initialization SKIPPED (need --extra
  522. 1078.60 s option to run) [ 36%]
  523. 1078.60 s tests/predict/test_knn.py::test_knn_query SKIPPED (need --extra option
  524. 1078.60 s to run) [ 36%]
  525. 1078.60 s tests/predict/test_knn.py::test_knn_query_specificity SKIPPED (need
  526. 1078.60 s --extra option to run) [ 36%]
  527. 1080.34 s tests/predict/test_multi_chain_comparison.py::test_basic_example PASSED [ 36%]
  528. 1082.14 s tests/predict/test_parallel.py::test_parallel_module PASSED [ 36%]
  529. 1083.83 s tests/predict/test_parallel.py::test_batch_module PASSED [ 36%]
  530. 1085.43 s tests/predict/test_parallel.py::test_nested_parallel_module PASSED [ 36%]
  531. 1087.09 s tests/predict/test_parallel.py::test_nested_batch_method PASSED [ 36%]
  532. 1088.74 s tests/predict/test_parallel.py::test_batch_with_failed_examples PASSED [ 36%]
  533. 1090.35 s tests/predict/test_parallel.py::test_parallel_timeout_and_straggler_limit_params PASSED [ 37%]
  534. 1092.06 s tests/predict/test_parallel.py::test_batch_timeout_and_straggler_limit_params PASSED [ 37%]
  535. 1093.72 s tests/predict/test_predict.py::test_initialization_with_string_signature PASSED [ 37%]
  536. 1095.38 s tests/predict/test_predict.py::test_reset_method PASSED [ 37%]
  537. 1097.23 s tests/predict/test_predict.py::test_lm_after_dump_and_load_state PASSED [ 37%]
  538. 1098.80 s tests/predict/test_predict.py::test_base_lm_dump_state_ignores_internal_class_marker_kwarg PASSED [ 37%]
  539. 1100.89 s tests/predict/test_predict.py::test_legacy_lm_state_without_class_marker_loads_as_lm PASSED [ 37%]
  540. 1102.54 s tests/predict/test_predict.py::test_custom_lm_load_state_requires_trusted_opt_in PASSED [ 37%]
  541. 1104.43 s tests/predict/test_predict.py::test_nested_custom_lm_class_path_loads_for_trusted_state PASSED [ 37%]
  542. 1106.07 s tests/predict/test_predict.py::test_call_method PASSED [ 37%]
  543. 1108.14 s tests/predict/test_predict.py::test_instructions_after_dump_and_load_state PASSED [ 37%]
  544. 1109.98 s tests/predict/test_predict.py::test_demos_after_dump_and_load_state PASSED [ 37%]
  545. 1111.70 s tests/predict/test_predict.py::test_typed_demos_after_dump_and_load_state PASSED [ 37%]
  546. 1113.92 s tests/predict/test_predict.py::test_signature_fields_after_dump_and_load_state PASSED [ 38%]
  547. 1116.17 s tests/predict/test_predict.py::test_lm_field_after_dump_and_load_state[model.json] PASSED [ 38%]
  548. 1118.28 s tests/predict/test_predict.py::test_lm_field_after_dump_and_load_state[model.pkl] PASSED [ 38%]
  549. 1120.25 s tests/predict/test_predict.py::test_load_ignores_serialized_endpoint_override_by_default[api_base] PASSED [ 38%]
  550. 1122.12 s tests/predict/test_predict.py::test_load_ignores_serialized_endpoint_override_by_default[base_url] PASSED [ 38%]
  551. 1124.21 s tests/predict/test_predict.py::test_load_allows_serialized_endpoint_override_with_opt_in[api_base] PASSED [ 38%]
  552. 1126.27 s tests/predict/test_predict.py::test_load_allows_serialized_endpoint_override_with_opt_in[base_url] PASSED [ 38%]
  553. 1128.33 s tests/predict/test_predict.py::test_load_state_ignores_serialized_endpoint_override_by_default[api_base] PASSED [ 38%]
  554. 1130.25 s tests/predict/test_predict.py::test_load_state_ignores_serialized_endpoint_override_by_default[base_url] PASSED [ 38%]
  555. 1132.42 s tests/predict/test_predict.py::test_load_state_allows_serialized_endpoint_override_with_opt_in[api_base] PASSED [ 38%]
  556. 1134.81 s tests/predict/test_predict.py::test_load_state_allows_serialized_endpoint_override_with_opt_in[base_url] PASSED [ 38%]
  557. 1137.13 s tests/predict/test_predict.py::test_load_state_ignores_serialized_model_list_endpoint_override_by_default PASSED [ 38%]
  558. 1139.17 s tests/predict/test_predict.py::test_load_prevents_serialized_endpoint_override_reaching_litellm[api_base] PASSED [ 38%]
  559. 1141.45 s tests/predict/test_predict.py::test_load_prevents_serialized_endpoint_override_reaching_litellm[base_url] PASSED [ 39%]
  560. 1143.42 s tests/predict/test_predict.py::test_load_blocks_serialized_model_list_unless_opted_in PASSED [ 39%]
  561. 1145.45 s tests/predict/test_predict.py::test_load_uses_env_api_key_without_honoring_serialized_endpoint_override PASSED [ 39%]
  562. 1148.12 s tests/predict/test_predict.py::test_forward_method PASSED [ 39%]
  563. 1150.06 s tests/predict/test_predict.py::test_forward_method2 PASSED [ 39%]
  564. 1151.82 s tests/predict/test_predict.py::test_config_management PASSED [ 39%]
  565. 1153.96 s tests/predict/test_predict.py::test_multi_output PASSED [ 39%]
  566. 1155.74 s tests/predict/test_predict.py::test_multi_output2 PASSED [ 39%]
  567. 1157.70 s tests/predict/test_predict.py::test_datetime_inputs_and_outputs PASSED [ 39%]
  568. 1159.73 s tests/predict/test_predict.py::test_explicitly_valued_enum_inputs_and_outputs PASSED [ 39%]
  569. 1161.62 s tests/predict/test_predict.py::test_enum_inputs_and_outputs_with_shared_names_and_values PASSED [ 39%]
  570. 1163.48 s tests/predict/test_predict.py::test_auto_valued_enum_inputs_and_outputs PASSED [ 39%]
  571. 1165.46 s tests/predict/test_predict.py::test_named_predictors PASSED [ 39%]
  572. 1167.26 s tests/predict/test_predict.py::test_output_only PASSED [ 40%]
  573. 1169.02 s tests/predict/test_predict.py::test_load_state_chaining PASSED [ 40%]
  574. 1170.85 s tests/predict/test_predict.py::test_call_predict_with_chat_history[chat] PASSED [ 40%]
  575. 1172.72 s tests/predict/test_predict.py::test_call_predict_with_chat_history[json] PASSED [ 40%]
  576. 1174.57 s tests/predict/test_predict.py::test_lm_usage PASSED [ 40%]
  577. 1177.07 s tests/predict/test_predict.py::test_lm_usage_with_parallel PASSED [ 40%]
  578. 1180.05 s tests/predict/test_predict.py::test_lm_usage_with_async PASSED [ 40%]
  579. 1182.03 s tests/predict/test_predict.py::test_positional_arguments PASSED [ 40%]
  580. 1183.90 s tests/predict/test_predict.py::test_error_message_on_invalid_lm_setup PASSED [ 40%]
  581. 1185.83 s tests/predict/test_predict.py::test_field_constraints[chat] PASSED [ 40%]
  582. 1187.63 s tests/predict/test_predict.py::test_field_constraints[json] PASSED [ 40%]
  583. 1189.61 s tests/predict/test_predict.py::test_async_predict PASSED [ 40%]
  584. 1191.59 s tests/predict/test_predict.py::test_predicted_outputs_piped_from_predict_to_lm_call PASSED [ 40%]
  585. 1193.52 s tests/predict/test_predict.py::test_dump_state_pydantic_non_primitive_types PASSED [ 40%]
  586. 1195.60 s tests/predict/test_predict.py::test_trace_size_limit PASSED [ 41%]
  587. 1197.66 s tests/predict/test_predict.py::test_disable_trace PASSED [ 41%]
  588. 1199.42 s tests/predict/test_predict.py::test_per_module_history_size_limit PASSED [ 41%]
  589. 1201.40 s tests/predict/test_predict.py::test_per_module_history_disabled PASSED [ 41%]
  590. 1203.22 s tests/predict/test_predict.py::test_input_field_default_value PASSED [ 41%]
  591. 1205.16 s tests/predict/test_predict.py::test_extra_fields_warning PASSED [ 41%]
  592. 1207.10 s tests/predict/test_predict.py::test_missing_optional_input_field_no_warning PASSED [ 41%]
  593. 1208.98 s tests/predict/test_predict.py::test_missing_required_input_field_still_warns PASSED [ 41%]
  594. 1210.91 s tests/predict/test_predict.py::test_warning_images PASSED [ 41%]
  595. 1212.88 s tests/predict/test_predict.py::test_type_mismatch_warning PASSED [ 41%]
  596. 1214.77 s tests/predict/test_predict.py::test_correct_types_no_warning PASSED [ 41%]
  597. 1216.49 s tests/predict/test_predict.py::test_list_type_validation PASSED [ 41%]
  598. 1218.24 s tests/predict/test_predict.py::test_literal_type_validation PASSED [ 41%]
  599. 1220.07 s tests/predict/test_predict.py::test_literal_union_type_validation PASSED [ 42%]
  600. 1221.88 s tests/predict/test_predict.py::test_list_string PASSED [ 42%]
  601. 1223.72 s tests/predict/test_predict.py::test_nested_list_type_validation PASSED [ 42%]
  602. 1225.46 s tests/predict/test_predict.py::test_nested_dict_type_validation PASSED [ 42%]
  603. 1227.23 s tests/predict/test_predict.py::test_nested_tuple_type_validation PASSED [ 42%]
  604. 1229.19 s tests/predict/test_predict.py::test_literal_type_validation_string_signature PASSED [ 42%]
  605. 1231.11 s tests/predict/test_predict.py::test_list_type_validation_string_signature PASSED [ 42%]
  606. 1232.79 s tests/predict/test_predict.py::test_dict_type_validation_string_signature PASSED [ 42%]
  607. 1234.58 s tests/predict/test_predict.py::test_tuple_type_validation_string_signature PASSED [ 42%]
  608. 1236.38 s tests/predict/test_predict.py::test_union_type_validation_string_signature PASSED [ 42%]
  609. 1238.14 s tests/predict/test_predict.py::test_basic_types_string_signature[False] PASSED [ 42%]
  610. 1239.93 s tests/predict/test_predict.py::test_basic_types_string_signature[True] PASSED [ 42%]
  611. 1241.73 s tests/predict/test_predict.py::test_untyped_string_signature PASSED [ 42%]
  612. 1243.53 s tests/predict/test_predict.py::test_untyped_class_signature PASSED [ 43%]
  613. 1245.25 s tests/predict/test_predict.py::test_string_to_list_signature PASSED [ 43%]
  614. 1247.00 s tests/predict/test_predict.py::test_custom_signature_types[False] PASSED [ 43%]
  615. 1248.82 s tests/predict/test_predict.py::test_custom_signature_types[True] PASSED [ 43%]
  616. 1248.82 s tests/predict/test_program_of_thought.py::test_pot_code_generation SKIPPED [ 43%]
  617. 1248.83 s tests/predict/test_program_of_thought.py::test_old_style_pot SKIPPED [ 43%]
  618. 1248.83 s tests/predict/test_program_of_thought.py::test_pot_support_multiple_fields SKIPPED [ 43%]
  619. 1248.83 s tests/predict/test_program_of_thought.py::test_pot_code_generation_with_one_error SKIPPED [ 43%]
  620. 1248.83 s tests/predict/test_program_of_thought.py::test_pot_code_generation_persistent_errors SKIPPED [ 43%]
  621. 1250.59 s tests/predict/test_program_of_thought.py::test_pot_code_parse_error PASSED [ 43%]
  622. 1250.59 s tests/predict/test_react.py::test_tool_observation_preserves_custom_type SKIPPED [ 43%]
  623. 1252.34 s tests/predict/test_react.py::test_tool_calling_with_pydantic_args PASSED [ 43%]
  624. 1254.28 s tests/predict/test_react.py::test_react_with_tools_skips_native_response_issubclass_for_generic_alias PASSED [ 43%]
  625. 1256.51 s tests/predict/test_react.py::test_tool_calling_without_typehint PASSED [ 44%]
  626. 1258.44 s tests/predict/test_react.py::test_trajectory_truncation PASSED [ 44%]
  627. 1260.27 s tests/predict/test_react.py::test_context_window_exceeded_after_retries PASSED [ 44%]
  628. 1262.17 s tests/predict/test_react.py::test_error_retry PASSED [ 44%]
  629. 1264.14 s tests/predict/test_react.py::test_async_tool_calling_with_pydantic_args PASSED [ 44%]
  630. 1266.07 s tests/predict/test_react.py::test_async_error_retry PASSED [ 44%]
  631. 1267.88 s tests/predict/test_react_v2.py::test_react_v2_submit_tool_returns_original_output_fields PASSED [ 44%]
  632. 1269.97 s tests/predict/test_react_v2.py::test_react_v2_text_mock_lm_loop_records_inputs_once PASSED [ 44%]
  633. 1272.06 s tests/predict/test_react_v2.py::test_react_v2_continuation_omits_missing_original_inputs PASSED [ 44%]
  634. 1273.87 s tests/predict/test_react_v2.py::test_react_v2_text_mode_accepts_top_level_tool_arguments PASSED [ 44%]
  635. 1275.77 s tests/predict/test_react_v2.py::test_react_v2_text_mode_accepts_wrapped_submit_arguments PASSED [ 44%]
  636. 1277.72 s tests/predict/test_react_v2.py::test_react_v2_unknown_tool_observation_can_continue PASSED [ 44%]
  637. 1279.77 s tests/predict/test_react_v2.py::test_react_v2_accepts_serialized_history_input PASSED [ 44%]
  638. 1281.55 s tests/predict/test_react_v2.py::test_react_v2_forced_submit_on_empty_tool_calls PASSED [ 45%]
  639. 1283.38 s tests/predict/test_react_v2.py::test_react_v2_native_tool_loop_replays_tool_result_with_provider_id PASSED [ 45%]
  640. 1285.31 s tests/predict/test_react_v2.py::test_react_v2_native_parallel_tool_calls_are_requested_and_replayed PASSED [ 45%]
  641. 1287.31 s tests/predict/test_refine.py::test_refine_forward_success_first_attempt PASSED [ 45%]
  642. 1289.20 s tests/predict/test_refine.py::test_refine_module_default_fail_count PASSED [ 45%]
  643. 1291.03 s tests/predict/test_refine.py::test_refine_module_custom_fail_count PASSED [ 45%]
  644. 1293.01 s tests/predict/test_rlm.py::TestMockInterpreter::test_scripted_responses PASSED [ 45%]
  645. 1294.99 s tests/predict/test_rlm.py::TestMockInterpreter::test_returns_final_output_result PASSED [ 45%]
  646. 1296.84 s tests/predict/test_rlm.py::TestMockInterpreter::test_raises_exception_from_responses PASSED [ 45%]
  647. 1298.82 s tests/predict/test_rlm.py::TestMockInterpreter::test_records_call_history PASSED [ 45%]
  648. 1300.85 s tests/predict/test_rlm.py::TestRLMInitialization::test_basic_initialization PASSED [ 45%]
  649. 1302.77 s tests/predict/test_rlm.py::TestRLMInitialization::test_custom_signature PASSED [ 45%]
  650. 1304.76 s tests/predict/test_rlm.py::TestRLMInitialization::test_custom_tools PASSED [ 45%]
  651. 1306.86 s tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_invalid_identifier[invalid-name] PASSED [ 45%]
  652. 1308.83 s tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_invalid_identifier[123start] PASSED [ 46%]
  653. 1310.75 s tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_reserved_names[llm_query] PASSED [ 46%]
  654. 1312.68 s tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_reserved_names[SUBMIT] PASSED [ 46%]
  655. 1314.69 s tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_reserved_names[print] PASSED [ 46%]
  656. 1316.57 s tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_not_callable[not a function] PASSED [ 46%]
  657. 1318.41 s tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_not_callable[123] PASSED [ 46%]
  658. 1320.40 s tests/predict/test_rlm.py::TestRLMInitialization::test_tools_dict_rejected PASSED [ 46%]
  659. 1322.40 s tests/predict/test_rlm.py::TestRLMInitialization::test_optional_parameters PASSED [ 46%]
  660. 1324.27 s tests/predict/test_rlm.py::TestRLMInitialization::test_forward_validates_required_inputs PASSED [ 46%]
  661. 1326.29 s tests/predict/test_rlm.py::TestRLMInitialization::test_batched_query_errors_have_clear_markers PASSED [ 46%]
  662. 1328.30 s tests/predict/test_rlm.py::TestRLMInitialization::test_tools_call_counter_is_thread_safe PASSED [ 46%]
  663. 1330.22 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```python\nprint(1)\n```-print(1)] PASSED [ 46%]
  664. 1332.13 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```py\nx = 1\nprint(x)\n```-x = 1\nprint(x)] PASSED [ 46%]
  665. 1334.34 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```\nprint('no lang')\n```-print('no lang')] PASSED [ 47%]
  666. 1336.49 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[not fenced code-not fenced code] PASSED [ 47%]
  667. 1339.14 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[I'll inspect first.\n```python\nprint('hello')\n```\nThen I will submit.-print('hello')] PASSED [ 47%]
  668. 1341.71 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```python\nprint(1)\n```\nsome trailing text-print(1)] PASSED [ 47%]
  669. 1344.07 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```python\nprint('oops')-print('oops')] PASSED [ 47%]
  670. 1346.49 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```\n```python\nprint(1)\n```\n```-print(1)] PASSED [ 47%]
  671. 1348.85 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```\n```\nprint(2)\n```\n```-print(2)] PASSED [ 47%]
  672. 1351.20 s tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences_rejects_non_python_lang PASSED [ 47%]
  673. 1353.87 s tests/predict/test_rlm.py::TestRLMFormatting::test_format_history PASSED [ 47%]
  674. 1356.14 s tests/predict/test_rlm.py::TestRLMFormatting::test_format_history_empty PASSED [ 47%]
  675. 1358.32 s tests/predict/test_rlm.py::TestRLMFormatting::test_action_signature_has_iteration_field PASSED [ 47%]
  676. 1360.63 s tests/predict/test_rlm.py::TestRLMFormatting::test_format_output PASSED [ 47%]
  677. 1362.73 s tests/predict/test_rlm.py::TestRLMFormatting::test_format_output_empty PASSED [ 47%]
  678. 1364.91 s tests/predict/test_rlm.py::TestRLMFormatting::test_format_output_passthrough PASSED [ 48%]
  679. 1367.07 s tests/predict/test_rlm.py::TestRLMFormatting::test_format_variable_info_string PASSED [ 48%]
  680. 1369.19 s tests/predict/test_rlm.py::TestRLMFormatting::test_format_variable_info_dict PASSED [ 48%]
  681. 1371.08 s tests/predict/test_rlm.py::TestRLMFormatting::test_build_variables_multiple PASSED [ 48%]
  682. 1373.12 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_history_immutability PASSED [ 48%]
  683. 1375.14 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_history_len_iter_bool PASSED [ 48%]
  684. 1377.07 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_entry_format PASSED [ 48%]
  685. 1379.06 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_entry_format_truncation PASSED [ 48%]
  686. 1381.12 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_entry_format_no_truncation PASSED [ 48%]
  687. 1383.11 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_history_threads_max_output_chars PASSED [ 48%]
  688. 1385.38 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_from_value PASSED [ 48%]
  689. 1388.42 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_truncation PASSED [ 48%]
  690. 1391.84 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_with_field_info PASSED [ 48%]
  691. 1395.39 s tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_without_field_info PASSED [ 49%]
  692. 1399.86 s tests/predict/test_rlm.py::TestREPLTypes::test_build_variables_includes_field_metadata PASSED [ 49%]
  693. 1403.86 s tests/predict/test_rlm.py::TestRLMCallMethod::test_call_is_alias_for_forward PASSED [ 49%]
  694. 1407.25 s tests/predict/test_rlm.py::TestRLMMaxIterationsFallback::test_max_iters_triggers_extract PASSED [ 49%]
  695. 1410.60 s tests/predict/test_rlm.py::TestRLMToolExceptions::test_tool_exception_returns_error_in_output PASSED [ 49%]
  696. 1413.51 s tests/predict/test_rlm.py::TestRLMToolExceptions::test_runtime_error_history_uses_stripped_code PASSED [ 49%]
  697. 1416.92 s tests/predict/test_rlm.py::TestRLMToolExceptions::test_syntax_error_from_execute_is_recoverable PASSED [ 49%]
  698. 1420.29 s tests/predict/test_rlm.py::TestRLMToolExceptions::test_syntax_error_from_strip_code_fences_is_recoverable PASSED [ 49%]
  699. 1423.70 s tests/predict/test_rlm.py::TestRLMDynamicSignature::test_action_signature_structure PASSED [ 49%]
  700. 1426.63 s tests/predict/test_rlm.py::TestRLMDynamicSignature::test_extract_signature_structure PASSED [ 49%]
  701. 1426.66 s tests/predict/test_rlm.py::TestPythonInterpreter::test_start_prewarms_sandbox SKIPPED [ 49%]
  702. 1426.66 s tests/predict/test_rlm.py::TestPythonInterpreter::test_start_is_idempotent SKIPPED [ 49%]
  703. 1426.66 s tests/predict/test_rlm.py::TestPythonInterpreter::test_basic_execution SKIPPED [ 49%]
  704. 1426.66 s tests/predict/test_rlm.py::TestPythonInterpreter::test_variable_injection SKIPPED [ 50%]
  705. 1426.66 s tests/predict/test_rlm.py::TestPythonInterpreter::test_variable_injection_with_none_values SKIPPED [ 50%]
  706. 1426.66 s tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_call_kwargs SKIPPED [ 50%]
  707. 1426.66 s tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_call_positional SKIPPED [ 50%]
  708. 1426.66 s tests/predict/test_rlm.py::TestPythonInterpreter::test_multiple_tools SKIPPED [ 50%]
  709. 1426.66 s tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_returns_list SKIPPED [ 50%]
  710. 1426.66 s tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_returns_dict SKIPPED [ 50%]
  711. 1426.66 s tests/predict/test_rlm.py::TestPythonInterpreter::test_state_persists SKIPPED [ 50%]
  712. 1426.66 s tests/predict/test_rlm.py::TestPythonInterpreter::test_syntax_error SKIPPED [ 50%]
  713. 1426.66 s tests/predict/test_rlm.py::TestPythonInterpreter::test_runtime_error SKIPPED [ 50%]
  714. 1426.66 s tests/predict/test_rlm.py::TestSandboxSecurity::test_no_network_access SKIPPED [ 50%]
  715. 1426.66 s tests/predict/test_rlm.py::TestSandboxSecurity::test_imports_work SKIPPED [ 50%]
  716. 1428.98 s tests/predict/test_rlm.py::TestRLMAsyncMock::test_aforward_basic PASSED [ 50%]
  717. 1431.28 s tests/predict/test_rlm.py::TestRLMAsyncMock::test_aforward_int_output_mock PASSED [ 50%]
  718. 1433.58 s tests/predict/test_rlm.py::TestRLMAsyncMock::test_aforward_multi_iteration_mock PASSED [ 51%]
  719. 1435.96 s tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[count-int-42-SUBMIT(42)-42] PASSED [ 51%]
  720. 1438.05 s tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[score-float-3.14-SUBMIT(3.14)-3.14] PASSED [ 51%]
  721. 1440.42 s tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[valid-bool-True-SUBMIT(True)-True] PASSED [ 51%]
  722. 1442.41 s tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[numbers-list[int]-final_value3-SUBMIT([1, 2, 3])-expected3] PASSED [ 51%]
  723. 1444.47 s tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[answer-Literal['yes', 'no']-yes-SUBMIT("yes")-yes] PASSED [ 51%]
  724. 1447.15 s tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_error_retries PASSED [ 51%]
  725. 1447.17 s tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[count-int-SUBMIT(42)-42-int] SKIPPED [ 51%]
  726. 1447.17 s tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[score-float-SUBMIT(3.14)-3.14-float] SKIPPED [ 51%]
  727. 1447.17 s tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[valid-bool-SUBMIT(True)-True-bool] SKIPPED [ 51%]
  728. 1447.17 s tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[numbers-list[int]-SUBMIT([1, 2, 3])-expected3-list] SKIPPED [ 51%]
  729. 1447.17 s tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[data-dict[str, str]-SUBMIT({"key": "value"})-expected4-dict] SKIPPED [ 51%]
  730. 1447.17 s tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[answer-Literal['yes', 'no']-SUBMIT("yes")-yes-str] SKIPPED [ 51%]
  731. 1447.17 s tests/predict/test_rlm.py::TestRLMTypeCoercion::test_submit_extracts_typed_value SKIPPED [ 52%]
  732. 1447.17 s tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_final_kwargs SKIPPED [ 52%]
  733. 1447.17 s tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_final_positional SKIPPED [ 52%]
  734. 1447.17 s tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_three_fields SKIPPED [ 52%]
  735. 1447.17 s tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_final_missing_field_errors SKIPPED [ 52%]
  736. 1447.17 s tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_submit_vars SKIPPED [ 52%]
  737. 1447.17 s tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_type_coercion SKIPPED [ 52%]
  738. 1447.17 s tests/predict/test_rlm.py::TestRLMWithDummyLM::test_simple_computation_e2e SKIPPED [ 52%]
  739. 1447.17 s tests/predict/test_rlm.py::TestRLMWithDummyLM::test_multi_turn_computation_e2e SKIPPED [ 52%]
  740. 1447.17 s tests/predict/test_rlm.py::TestRLMWithDummyLM::test_with_input_variables_e2e SKIPPED [ 52%]
  741. 1447.17 s tests/predict/test_rlm.py::TestRLMWithDummyLM::test_with_tool_e2e SKIPPED [ 52%]
  742. 1447.17 s tests/predict/test_rlm.py::TestRLMWithDummyLM::test_aforward_simple_computation_e2e SKIPPED [ 52%]
  743. 1447.17 s tests/predict/test_rlm.py::TestRLMWithDummyLM::test_aforward_multi_turn_e2e SKIPPED [ 52%]
  744. 1447.17 s tests/predict/test_rlm.py::TestRLMWithDummyLM::test_aforward_with_input_variables_e2e SKIPPED [ 53%]
  745. 1447.17 s tests/predict/test_rlm.py::TestRLMIntegration::test_simple_computation SKIPPED [ 53%]
  746. 1447.17 s tests/predict/test_rlm.py::TestRLMIntegration::test_with_llm_query SKIPPED [ 53%]
  747. 1450.04 s tests/predict/test_rlm.py::TestBuildVariablesWithSerializable::test_serializable_uses_build_repl_variable PASSED [ 53%]
  748. 1452.98 s tests/predict/test_rlm.py::TestBuildVariablesWithSerializable::test_regular_values_unchanged PASSED [ 53%]
  749. 1455.80 s tests/predict/test_rlm.py::TestPrepareSerializableVars::test_separates_serializable_from_regular PASSED [ 53%]
  750. 1459.24 s tests/predict/test_rlm.py::TestPrepareSerializableVars::test_no_serializable_returns_all PASSED [ 53%]
  751. 1462.45 s tests/predict/test_rlm.py::TestPrepareSerializableVars::test_binary_payload_uses_base64_transport PASSED [ 53%]
  752. 1465.08 s tests/predict/test_rlm.py::TestPrepareSerializableVars::test_large_payload_not_inlined_in_code PASSED [ 53%]
  753. 1467.08 s tests/predict/test_rlm.py::TestPrepareSerializableVars::test_forward_with_serializable PASSED [ 53%]
  754. 1467.08 s tests/predict/test_rlm.py::TestLargeSerializableRoundTrip::test_large_payload_round_trips_through_real_sandbox SKIPPED [ 53%]
  755. 1469.09 s tests/primitives/test_base_module.py::test_deepcopy_basic PASSED [ 53%]
  756. 1471.17 s tests/primitives/test_base_module.py::test_deepcopy_with_uncopyable_modules PASSED [ 53%]
  757. 1473.20 s tests/primitives/test_base_module.py::test_deepcopy_with_nested_modules PASSED [ 54%]
  758. 1475.31 s tests/primitives/test_base_module.py::test_save_and_load_with_json PASSED [ 54%]
  759. 1475.31 s tests/primitives/test_base_module.py::test_save_and_load_with_pkl SKIPPED [ 54%]
  760. 1477.34 s tests/primitives/test_base_module.py::test_save_with_extra_modules PASSED [ 54%]
  761. 1479.48 s tests/primitives/test_base_module.py::test_load_with_version_mismatch PASSED [ 54%]
  762. 1479.48 s tests/primitives/test_base_module.py::test_single_module_call_with_usage_tracker SKIPPED [ 54%]
  763. 1479.48 s tests/primitives/test_base_module.py::test_multi_module_call_with_usage_tracker SKIPPED [ 54%]
  764. 1479.48 s tests/primitives/test_base_module.py::test_usage_tracker_in_parallel SKIPPED [ 54%]
  765. 1481.80 s tests/primitives/test_base_module.py::test_usage_tracker_async_parallel PASSED [ 54%]
  766. 1483.94 s tests/primitives/test_base_module.py::test_usage_tracker_no_side_effect PASSED [ 54%]
  767. 1486.25 s tests/primitives/test_base_module.py::test_module_history PASSED [ 54%]
  768. 1488.32 s tests/primitives/test_base_module.py::test_module_history_with_concurrency PASSED [ 54%]
  769. 1490.66 s tests/primitives/test_base_module.py::test_module_history_async PASSED [ 54%]
  770. 1492.91 s tests/primitives/test_base_module.py::test_forward_direct_call_warning PASSED [ 54%]
  771. 1495.15 s tests/primitives/test_base_module.py::test_forward_through_call_no_warning PASSED [ 55%]
  772. 1497.23 s tests/primitives/test_example.py::test_example_initialization PASSED [ 55%]
  773. 1499.79 s tests/primitives/test_example.py::test_example_initialization_from_base PASSED [ 55%]
  774. 1503.32 s tests/primitives/test_example.py::test_example_initialization_from_dict PASSED [ 55%]
  775. 1507.94 s tests/primitives/test_example.py::test_example_set_get_item PASSED [ 55%]
  776. 1512.56 s tests/primitives/test_example.py::test_example_attribute_access PASSED [ 55%]
  777. 1517.58 s tests/primitives/test_example.py::test_example_deletion
  778. 1518.17 s PASSED [ 55%]
  779. 1523.18 s tests/primitives/test_example.py::test_example_len
  780. 1523.23 s PASSED [ 55%]
  781. 1526.85 s tests/primitives/test_example.py::test_example_repr_str_img PASSED [ 55%]
  782. 1530.48 s tests/primitives/test_example.py::test_example_repr_str PASSED [ 55%]
  783. 1533.79 s tests/primitives/test_example.py::test_example_eq PASSED [ 55%]
  784. 1536.94 s tests/primitives/test_example.py::test_example_hash PASSED [ 55%]
  785. 1539.99 s tests/primitives/test_example.py::test_example_keys_values_items PASSED [ 55%]
  786. 1543.10 s tests/primitives/test_example.py::test_example_get PASSED [ 56%]
  787. 1545.45 s tests/primitives/test_example.py::test_example_with_inputs PASSED [ 56%]
  788. 1547.87 s tests/primitives/test_example.py::test_example_inputs_labels PASSED [ 56%]
  789. 1550.26 s tests/primitives/test_example.py::test_example_copy_without PASSED [ 56%]
  790. 1552.86 s tests/primitives/test_example.py::test_example_to_dict PASSED [ 56%]
  791. 1555.40 s tests/primitives/test_example.py::test_example_to_dict_with_history PASSED [ 56%]
  792. 1557.58 s tests/primitives/test_module.py::test_module_initialization PASSED [ 56%]
  793. 1560.23 s tests/primitives/test_module.py::test_named_predictors PASSED [ 56%]
  794. 1562.47 s tests/primitives/test_module.py::test_predictors PASSED [ 56%]
  795. 1565.06 s tests/primitives/test_module.py::test_forward PASSED [ 56%]
  796. 1567.38 s tests/primitives/test_module.py::test_nested_named_predictors PASSED [ 56%]
  797. 1569.75 s tests/primitives/test_module.py::test_empty_module PASSED [ 56%]
  798. 1573.29 s tests/primitives/test_module.py::test_single_level PASSED [ 56%]
  799. 1576.99 s tests/primitives/test_module.py::test_multiple_levels PASSED [ 57%]
  800. 1579.21 s tests/primitives/test_module.py::test_multiple_sub_modules PASSED [ 57%]
  801. 1582.12 s tests/primitives/test_module.py::test_non_base_module_attributes PASSED [ 57%]
  802. 1584.59 s tests/primitives/test_module.py::test_complex_module_traversal PASSED [ 57%]
  803. 1587.35 s tests/primitives/test_module.py::test_complex_module_traversal_with_same_module PASSED [ 57%]
  804. 1590.86 s tests/primitives/test_module.py::test_complex_module_set_attribute_by_name PASSED [ 57%]
  805. 1593.88 s tests/primitives/test_module.py::test_named_parameters_duplicate_references PASSED [ 57%]
  806. 1596.59 s tests/primitives/test_module.py::test_load_dspy_program_cross_version PASSED [ 57%]
  807. 1599.43 s tests/primitives/test_module.py::test_load_state_is_transactional PASSED [ 57%]
  808. 1599.45 s tests/primitives/test_python_interpreter.py::test_execute_simple_code SKIPPED [ 57%]
  809. 1599.45 s tests/primitives/test_python_interpreter.py::test_import SKIPPED (need
  810. 1599.45 s --deno option to run) [ 57%]
  811. 1599.45 s tests/primitives/test_python_interpreter.py::test_user_variable_definitions SKIPPED [ 57%]
  812. 1599.45 s tests/primitives/test_python_interpreter.py::test_rejects_python_keywords_as_variable_names SKIPPED [ 57%]
  813. 1599.45 s tests/primitives/test_python_interpreter.py::test_failure_syntax_error SKIPPED [ 58%]
  814. 1599.45 s tests/primitives/test_python_interpreter.py::test_failure_zero_division SKIPPED [ 58%]
  815. 1599.45 s tests/primitives/test_python_interpreter.py::test_exception_args SKIPPED [ 58%]
  816. 1599.45 s tests/primitives/test_python_interpreter.py::test_submit_with_list SKIPPED [ 58%]
  817. 1599.45 s tests/primitives/test_python_interpreter.py::test_enable_env_vars_flag SKIPPED [ 58%]
  818. 1599.45 s tests/primitives/test_python_interpreter.py::test_read_file_access_control SKIPPED [ 58%]
  819. 1599.45 s tests/primitives/test_python_interpreter.py::test_enable_write_flag SKIPPED [ 58%]
  820. 1599.45 s tests/primitives/test_python_interpreter.py::test_enable_net_flag SKIPPED [ 58%]
  821. 1599.45 s tests/primitives/test_python_interpreter.py::test_interpreter_security_filesystem_access SKIPPED [ 58%]
  822. 1599.45 s tests/primitives/test_python_interpreter.py::test_tools_dict_is_copied SKIPPED [ 58%]
  823. 1599.45 s tests/primitives/test_python_interpreter.py::test_serialize_tuple SKIPPED [ 58%]
  824. 1599.45 s tests/primitives/test_python_interpreter.py::test_serialize_set SKIPPED [ 58%]
  825. 1599.45 s tests/primitives/test_python_interpreter.py::test_serialize_set_mixed_types SKIPPED [ 58%]
  826. 1599.45 s tests/primitives/test_python_interpreter.py::test_deno_command_dict_raises_type_error SKIPPED [ 59%]
  827. 1599.45 s tests/primitives/test_python_interpreter.py::test_tool_with_typed_signature SKIPPED [ 59%]
  828. 1599.45 s tests/primitives/test_python_interpreter.py::test_tool_positional_args SKIPPED [ 59%]
  829. 1599.45 s tests/primitives/test_python_interpreter.py::test_tool_keyword_args SKIPPED [ 59%]
  830. 1599.45 s tests/primitives/test_python_interpreter.py::test_tool_default_args SKIPPED [ 59%]
  831. 1599.45 s tests/primitives/test_python_interpreter.py::test_tools_re_register_after_process_restart SKIPPED [ 59%]
  832. 1599.45 s tests/primitives/test_python_interpreter.py::test_mounts_replay_after_process_restart SKIPPED [ 59%]
  833. 1599.45 s tests/primitives/test_python_interpreter.py::test_tool_all_positional_args SKIPPED [ 59%]
  834. 1599.45 s tests/primitives/test_python_interpreter.py::test_tool_error_surfaces_as_runtime_error SKIPPED [ 59%]
  835. 1599.45 s tests/primitives/test_python_interpreter.py::test_tool_async_def_function SKIPPED [ 59%]
  836. 1599.45 s tests/primitives/test_python_interpreter.py::test_tool_async_def_raises_propagates SKIPPED [ 59%]
  837. 1599.45 s tests/primitives/test_python_interpreter.py::test_submit_with_typed_signature SKIPPED [ 59%]
  838. 1599.45 s tests/primitives/test_python_interpreter.py::test_submit_positional_args SKIPPED [ 59%]
  839. 1599.45 s tests/primitives/test_python_interpreter.py::test_submit_multi_output SKIPPED [ 59%]
  840. 1599.45 s tests/primitives/test_python_interpreter.py::test_submit_wrong_arg_count SKIPPED [ 60%]
  841. 1599.45 s tests/primitives/test_python_interpreter.py::test_extract_parameters SKIPPED [ 60%]
  842. 1599.45 s tests/primitives/test_python_interpreter.py::test_extract_parameters_complex_types SKIPPED [ 60%]
  843. 1599.45 s tests/primitives/test_python_interpreter.py::test_large_variable_injection SKIPPED [ 60%]
  844. 1599.45 s tests/primitives/test_python_interpreter.py::test_large_variable_content_integrity SKIPPED [ 60%]
  845. 1599.45 s tests/primitives/test_python_interpreter.py::test_mixed_small_and_large_variables SKIPPED [ 60%]
  846. 1599.45 s tests/primitives/test_python_interpreter.py::test_multiple_large_variables SKIPPED [ 60%]
  847. 1599.45 s tests/primitives/test_python_interpreter.py::test_large_list_variable SKIPPED [ 60%]
  848. 1599.45 s tests/primitives/test_python_interpreter.py::test_nested_sets_and_tuples SKIPPED [ 60%]
  849. 1599.45 s tests/primitives/test_python_interpreter.py::test_small_variable_not_using_filesystem SKIPPED [ 60%]
  850. 1599.45 s tests/primitives/test_python_interpreter.py::test_large_variable_threshold_boundary SKIPPED [ 60%]
  851. 1599.45 s tests/primitives/test_python_interpreter.py::test_enable_read_paths_symlink SKIPPED [ 60%]
  852. 1599.45 s tests/primitives/test_python_interpreter.py::test_enable_read_paths_multiple_files SKIPPED [ 60%]
  853. 1602.41 s tests/primitives/test_sandbox_serializable.py::TestABCConformance::test_subclass_conformance PASSED [ 61%]
  854. 1605.53 s tests/primitives/test_sandbox_serializable.py::TestABCConformance::test_incomplete_subclass_cannot_instantiate PASSED [ 61%]
  855. 1608.46 s tests/primitives/test_sandbox_serializable.py::TestABCConformance::test_structural_conformance_no_longer_accepted PASSED [ 61%]
  856. 1611.45 s tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_sandbox_setup PASSED [ 61%]
  857. 1613.63 s tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_to_sandbox_returns_bytes PASSED [ 61%]
  858. 1615.87 s tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_sandbox_assignment PASSED [ 61%]
  859. 1618.66 s tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_rlm_preview PASSED [ 61%]
  860. 1621.17 s tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_rlm_preview_truncation PASSED [ 61%]
  861. 1623.45 s tests/primitives/test_sandbox_serializable.py::TestBuildReplVariable::test_builds_variable_with_preview_from_rlm_preview PASSED [ 61%]
  862. 1625.67 s tests/primitives/test_sandbox_serializable.py::TestBuildReplVariable::test_surfaces_sandbox_setup_in_description PASSED [ 61%]
  863. 1628.02 s tests/primitives/test_sandbox_serializable.py::TestBuildReplVariable::test_passes_field_info_through PASSED [ 61%]
  864. 1630.39 s tests/primitives/test_sandbox_serializable.py::TestToReplVariableMethod::test_default_to_repl_variable PASSED [ 61%]
  865. 1632.61 s tests/primitives/test_sandbox_serializable.py::TestSignatureAnnotation::test_subclass_supports_signature_annotation PASSED [ 61%]
  866. 1634.76 s tests/propose/test_grounded_proposer.py::test_propose_instructions_for_program[None] PASSED [ 62%]
  867. 1636.93 s tests/propose/test_grounded_proposer.py::test_propose_instructions_for_program[demo_candidates1] PASSED [ 62%]
  868. 1639.20 s tests/propose/test_grounded_proposer.py::test_propose_instruction_for_predictor[None] PASSED [ 62%]
  869. 1641.46 s tests/propose/test_grounded_proposer.py::test_propose_instruction_for_predictor[demo_candidates1] PASSED [ 62%]
  870. 1641.46 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_many_types_1-input1] SKIPPED [ 62%]
  871. 1641.46 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_many_types_1-input2] SKIPPED [ 62%]
  872. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_nesting_1-input1] SKIPPED [ 62%]
  873. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_nesting_1-input2] SKIPPED [ 62%]
  874. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_nesting_2-input1] SKIPPED [ 62%]
  875. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-input_formats-test_markdown_1-input1] SKIPPED [ 62%]
  876. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-input_formats-test_markdown_1-input2] SKIPPED [ 62%]
  877. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_many_types_1-input1] SKIPPED [ 62%]
  878. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_many_types_1-input2] SKIPPED [ 62%]
  879. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_nesting_1-input1] SKIPPED [ 63%]
  880. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_nesting_1-input2] SKIPPED [ 63%]
  881. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_nesting_2-input1] SKIPPED [ 63%]
  882. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-input_formats-test_markdown_1-input1] SKIPPED [ 63%]
  883. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-input_formats-test_markdown_1-input2] SKIPPED [ 63%]
  884. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_many_types_1-input1] SKIPPED [ 63%]
  885. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_many_types_1-input2] SKIPPED [ 63%]
  886. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_nesting_1-input1] SKIPPED [ 63%]
  887. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_nesting_1-input2] SKIPPED [ 63%]
  888. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_nesting_2-input1] SKIPPED [ 63%]
  889. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-input_formats-test_markdown_1-input1] SKIPPED [ 63%]
  890. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-input_formats-test_markdown_1-input2] SKIPPED [ 63%]
  891. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_many_types_1-input1] SKIPPED [ 63%]
  892. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_many_types_1-input2] SKIPPED [ 63%]
  893. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_nesting_1-input1] SKIPPED [ 64%]
  894. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_nesting_1-input2] SKIPPED [ 64%]
  895. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_nesting_2-input1] SKIPPED [ 64%]
  896. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-input_formats-test_markdown_1-input1] SKIPPED [ 64%]
  897. 1641.47 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-input_formats-test_markdown_1-input2] SKIPPED [ 64%]
  898. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_many_types_1-input1] SKIPPED [ 64%]
  899. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_many_types_1-input2] SKIPPED [ 64%]
  900. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_nesting_1-input1] SKIPPED [ 64%]
  901. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_nesting_1-input2] SKIPPED [ 64%]
  902. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_nesting_2-input1] SKIPPED [ 64%]
  903. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-input_formats-test_markdown_1-input1] SKIPPED [ 64%]
  904. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-input_formats-test_markdown_1-input2] SKIPPED [ 64%]
  905. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_many_types_1-input1] SKIPPED [ 64%]
  906. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_many_types_1-input2] SKIPPED [ 65%]
  907. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_nesting_1-input1] SKIPPED [ 65%]
  908. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_nesting_1-input2] SKIPPED [ 65%]
  909. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_nesting_2-input1] SKIPPED [ 65%]
  910. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-input_formats-test_markdown_1-input1] SKIPPED [ 65%]
  911. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-input_formats-test_markdown_1-input2] SKIPPED [ 65%]
  912. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_many_types_1-input1] SKIPPED [ 65%]
  913. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_many_types_1-input2] SKIPPED [ 65%]
  914. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_nesting_1-input1] SKIPPED [ 65%]
  915. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_nesting_1-input2] SKIPPED [ 65%]
  916. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_nesting_2-input1] SKIPPED [ 65%]
  917. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-input_formats-test_markdown_1-input1] SKIPPED [ 65%]
  918. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-input_formats-test_markdown_1-input2] SKIPPED [ 65%]
  919. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_many_types_1-input1] SKIPPED [ 66%]
  920. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_many_types_1-input2] SKIPPED [ 66%]
  921. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_nesting_1-input1] SKIPPED [ 66%]
  922. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_nesting_1-input2] SKIPPED [ 66%]
  923. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_nesting_2-input1] SKIPPED [ 66%]
  924. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-input_formats-test_markdown_1-input1] SKIPPED [ 66%]
  925. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-input_formats-test_markdown_1-input2] SKIPPED [ 66%]
  926. 1641.58 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_many_types_1-input1] SKIPPED [ 66%]
  927. 1641.70 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_many_types_1-input2] SKIPPED [ 66%]
  928. 1641.70 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_nesting_1-input1] SKIPPED [ 66%]
  929. 1641.70 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_nesting_1-input2] SKIPPED [ 66%]
  930. 1641.70 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_nesting_2-input1] SKIPPED [ 66%]
  931. 1641.70 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-input_formats-test_markdown_1-input1] SKIPPED [ 66%]
  932. 1641.71 s tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-input_formats-test_markdown_1-input2] SKIPPED [ 67%]
  933. 1641.71 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 67%]
  934. 1641.71 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 67%]
  935. 1641.71 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 67%]
  936. 1641.71 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 67%]
  937. 1641.74 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 67%]
  938. 1641.74 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 67%]
  939. 1641.74 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 67%]
  940. 1641.74 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 67%]
  941. 1641.74 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 67%]
  942. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 67%]
  943. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 67%]
  944. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 67%]
  945. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 68%]
  946. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 68%]
  947. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 68%]
  948. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 68%]
  949. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 68%]
  950. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 68%]
  951. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 68%]
  952. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 68%]
  953. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 68%]
  954. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 68%]
  955. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 68%]
  956. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 68%]
  957. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 68%]
  958. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 68%]
  959. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 69%]
  960. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 69%]
  961. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_many_types_1-input1] SKIPPED [ 69%]
  962. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_many_types_1-input2] SKIPPED [ 69%]
  963. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_nesting_1-input1] SKIPPED [ 69%]
  964. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_nesting_1-input2] SKIPPED [ 69%]
  965. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_nesting_2-input1] SKIPPED [ 69%]
  966. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-input_formats-test_markdown_1-input1] SKIPPED [ 69%]
  967. 1641.76 s tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-input_formats-test_markdown_1-input2] SKIPPED [ 69%]
  968. 1641.76 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-4o] SKIPPED [ 69%]
  969. 1641.76 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-4o-mini] SKIPPED [ 69%]
  970. 1641.76 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-4-turbo] SKIPPED [ 69%]
  971. 1641.76 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-o1-preview] SKIPPED [ 69%]
  972. 1641.76 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-o1-mini] SKIPPED [ 70%]
  973. 1641.76 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[claude-3.5-sonnet] SKIPPED [ 70%]
  974. 1641.76 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[claude-3.5-haiku] SKIPPED [ 70%]
  975. 1641.76 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gemini-1.5-pro] SKIPPED [ 70%]
  976. 1641.76 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gemini-1.5-flash] SKIPPED [ 70%]
  977. 1641.76 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.1-405b-instruct] SKIPPED [ 70%]
  978. 1641.76 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.1-70b-instruct] SKIPPED [ 70%]
  979. 1641.76 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.1-8b-instruct] SKIPPED [ 70%]
  980. 1641.76 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.2-3b-instruct] SKIPPED [ 70%]
  981. 1641.76 s tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[deepseek-r1] SKIPPED [ 70%]
  982. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-Predict] SKIPPED [ 70%]
  983. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-ChainOfThought] SKIPPED [ 70%]
  984. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-mini-Predict] SKIPPED [ 70%]
  985. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-mini-ChainOfThought] SKIPPED [ 71%]
  986. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4-turbo-Predict] SKIPPED [ 71%]
  987. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4-turbo-ChainOfThought] SKIPPED [ 71%]
  988. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-preview-Predict] SKIPPED [ 71%]
  989. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-preview-ChainOfThought] SKIPPED [ 71%]
  990. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-mini-Predict] SKIPPED [ 71%]
  991. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-mini-ChainOfThought] SKIPPED [ 71%]
  992. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-sonnet-Predict] SKIPPED [ 71%]
  993. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-sonnet-ChainOfThought] SKIPPED [ 71%]
  994. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-haiku-Predict] SKIPPED [ 71%]
  995. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-haiku-ChainOfThought] SKIPPED [ 71%]
  996. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-pro-Predict] SKIPPED [ 71%]
  997. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-pro-ChainOfThought] SKIPPED [ 71%]
  998. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-flash-Predict] SKIPPED [ 72%]
  999. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-flash-ChainOfThought] SKIPPED [ 72%]
  1000. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-405b-instruct-Predict] SKIPPED [ 72%]
  1001. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-405b-instruct-ChainOfThought] SKIPPED [ 72%]
  1002. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-70b-instruct-Predict] SKIPPED [ 72%]
  1003. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-70b-instruct-ChainOfThought] SKIPPED [ 72%]
  1004. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-8b-instruct-Predict] SKIPPED [ 72%]
  1005. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-8b-instruct-ChainOfThought] SKIPPED [ 72%]
  1006. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.2-3b-instruct-Predict] SKIPPED [ 72%]
  1007. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.2-3b-instruct-ChainOfThought] SKIPPED [ 72%]
  1008. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[deepseek-r1-Predict] SKIPPED [ 72%]
  1009. 1641.76 s tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[deepseek-r1-ChainOfThought] SKIPPED [ 72%]
  1010. 1641.76 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-4o] SKIPPED [ 72%]
  1011. 1641.76 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-4o-mini] SKIPPED [ 72%]
  1012. 1641.76 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-4-turbo] SKIPPED [ 73%]
  1013. 1641.77 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-o1-preview] SKIPPED [ 73%]
  1014. 1641.77 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-o1-mini] SKIPPED [ 73%]
  1015. 1641.77 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[claude-3.5-sonnet] SKIPPED [ 73%]
  1016. 1641.77 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[claude-3.5-haiku] SKIPPED [ 73%]
  1017. 1641.77 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gemini-1.5-pro] SKIPPED [ 73%]
  1018. 1641.77 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gemini-1.5-flash] SKIPPED [ 73%]
  1019. 1641.77 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.1-405b-instruct] SKIPPED [ 73%]
  1020. 1641.77 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.1-70b-instruct] SKIPPED [ 73%]
  1021. 1641.77 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.1-8b-instruct] SKIPPED [ 73%]
  1022. 1641.77 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.2-3b-instruct] SKIPPED [ 73%]
  1023. 1641.77 s tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[deepseek-r1] SKIPPED [ 73%]
  1024. 1641.77 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-Predict] SKIPPED [ 73%]
  1025. 1641.77 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-ChainOfThought] SKIPPED [ 74%]
  1026. 1641.77 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-mini-Predict] SKIPPED [ 74%]
  1027. 1641.77 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-mini-ChainOfThought] SKIPPED [ 74%]
  1028. 1641.77 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4-turbo-Predict] SKIPPED [ 74%]
  1029. 1641.77 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4-turbo-ChainOfThought] SKIPPED [ 74%]
  1030. 1641.77 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-preview-Predict] SKIPPED [ 74%]
  1031. 1641.77 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-preview-ChainOfThought] SKIPPED [ 74%]
  1032. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-mini-Predict] SKIPPED [ 74%]
  1033. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-mini-ChainOfThought] SKIPPED [ 74%]
  1034. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-sonnet-Predict] SKIPPED [ 74%]
  1035. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-sonnet-ChainOfThought] SKIPPED [ 74%]
  1036. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-haiku-Predict] SKIPPED [ 74%]
  1037. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-haiku-ChainOfThought] SKIPPED [ 74%]
  1038. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-pro-Predict] SKIPPED [ 75%]
  1039. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-pro-ChainOfThought] SKIPPED [ 75%]
  1040. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-flash-Predict] SKIPPED [ 75%]
  1041. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-flash-ChainOfThought] SKIPPED [ 75%]
  1042. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-405b-instruct-Predict] SKIPPED [ 75%]
  1043. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-405b-instruct-ChainOfThought] SKIPPED [ 75%]
  1044. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-70b-instruct-Predict] SKIPPED [ 75%]
  1045. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-70b-instruct-ChainOfThought] SKIPPED [ 75%]
  1046. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-8b-instruct-Predict] SKIPPED [ 75%]
  1047. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-8b-instruct-ChainOfThought] SKIPPED [ 75%]
  1048. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.2-3b-instruct-Predict] SKIPPED [ 75%]
  1049. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.2-3b-instruct-ChainOfThought] SKIPPED [ 75%]
  1050. 1641.78 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[deepseek-r1-Predict] SKIPPED [ 75%]
  1051. 1641.79 s tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[deepseek-r1-ChainOfThought] SKIPPED [ 76%]
  1052. 1644.01 s tests/retrievers/test_colbertv2.py::test_get_request_raises_on_server_error PASSED [ 76%]
  1053. 1646.20 s tests/retrievers/test_colbertv2.py::test_post_request_raises_on_server_error PASSED [ 76%]
  1054. 1648.46 s tests/retrievers/test_colbertv2.py::test_get_request_success PASSED [ 76%]
  1055. 1650.92 s tests/retrievers/test_colbertv2.py::test_post_request_success PASSED [ 76%]
  1056. 1653.32 s tests/retrievers/test_embeddings.py::test_embeddings_basic_search PASSED [ 76%]
  1057. 1655.88 s tests/retrievers/test_embeddings.py::test_embeddings_multithreaded_search PASSED [ 76%]
  1058. 1658.31 s tests/retrievers/test_embeddings.py::test_embeddings_save_load PASSED [ 76%]
  1059. 1660.47 s tests/retrievers/test_embeddings.py::test_embeddings_from_saved PASSED [ 76%]
  1060. 1662.74 s tests/retrievers/test_embeddings.py::test_embeddings_load_nonexistent_path PASSED [ 76%]
  1061. 1664.79 s tests/retrievers/test_embeddings.py::test_embeddings_with_scores_basic_search PASSED [ 76%]
  1062. 1667.31 s tests/retrievers/test_embeddings.py::test_embeddings_with_scores_save_load PASSED [ 76%]
  1063. 1669.62 s tests/signatures/test_adapter_file.py::test_file_from_local_path PASSED [ 76%]
  1064. 1671.98 s tests/signatures/test_adapter_file.py::test_file_from_path_method PASSED [ 77%]
  1065. 1674.63 s tests/signatures/test_adapter_file.py::test_file_from_path_with_custom_filename PASSED [ 77%]
  1066. 1677.60 s tests/signatures/test_adapter_file.py::test_file_from_bytes PASSED [ 77%]
  1067. 1680.63 s tests/signatures/test_adapter_file.py::test_file_from_bytes_with_filename PASSED [ 77%]
  1068. 1683.92 s tests/signatures/test_adapter_file.py::test_file_from_file_id PASSED [ 77%]
  1069. 1687.33 s tests/signatures/test_adapter_file.py::test_file_from_file_id_with_filename PASSED [ 77%]
  1070. 1691.14 s tests/signatures/test_adapter_file.py::test_file_from_dict_with_file_data PASSED [ 77%]
  1071. 1694.47 s tests/signatures/test_adapter_file.py::test_file_from_dict_with_file_id PASSED [ 77%]
  1072. 1697.70 s tests/signatures/test_adapter_file.py::test_file_format_with_file_data PASSED [ 77%]
  1073. 1700.87 s tests/signatures/test_adapter_file.py::test_file_format_with_file_id PASSED [ 77%]
  1074. 1704.11 s tests/signatures/test_adapter_file.py::test_file_repr_with_file_data PASSED [ 77%]
  1075. 1706.43 s tests/signatures/test_adapter_file.py::test_file_repr_with_file_id PASSED [ 77%]
  1076. 1708.99 s tests/signatures/test_adapter_file.py::test_file_str PASSED [ 77%]
  1077. 1711.04 s tests/signatures/test_adapter_file.py::test_encode_file_to_dict_from_path PASSED [ 77%]
  1078. 1713.00 s tests/signatures/test_adapter_file.py::test_encode_file_to_dict_from_bytes PASSED [ 78%]
  1079. 1715.09 s tests/signatures/test_adapter_file.py::test_invalid_file_string PASSED [ 78%]
  1080. 1717.03 s tests/signatures/test_adapter_file.py::test_invalid_dict PASSED [ 78%]
  1081. 1718.93 s tests/signatures/test_adapter_file.py::test_file_in_signature PASSED [ 78%]
  1082. 1721.15 s tests/signatures/test_adapter_file.py::test_file_list_in_signature PASSED [ 78%]
  1083. 1723.32 s tests/signatures/test_adapter_file.py::test_optional_file_field PASSED [ 78%]
  1084. 1725.44 s tests/signatures/test_adapter_file.py::test_save_load_file_signature PASSED [ 78%]
  1085. 1727.53 s tests/signatures/test_adapter_file.py::test_file_frozen PASSED [ 78%]
  1086. 1729.34 s tests/signatures/test_adapter_file.py::test_file_with_all_fields PASSED [ 78%]
  1087. 1731.36 s tests/signatures/test_adapter_file.py::test_file_path_not_found PASSED [ 78%]
  1088. 1733.56 s tests/signatures/test_adapter_file.py::test_file_custom_mime_type PASSED [ 78%]
  1089. 1735.39 s tests/signatures/test_adapter_file.py::test_file_from_bytes_custom_mime PASSED [ 78%]
  1090. 1737.39 s tests/signatures/test_adapter_file.py::test_file_data_uri_in_format PASSED [ 78%]
  1091. 1739.35 s tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case0] PASSED [ 79%]
  1092. 1741.37 s tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case1] PASSED [ 79%]
  1093. 1743.21 s tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case2] PASSED [ 79%]
  1094. 1744.94 s tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case3] PASSED [ 79%]
  1095. 1746.84 s tests/signatures/test_adapter_image.py::test_image_input_formats[pil_image-PIL Image] XFAIL [ 79%]
  1096. 1748.84 s tests/signatures/test_adapter_image.py::test_image_input_formats[encoded_pil_image-encoded PIL image string] XFAILon) [ 79%]
  1097. 1750.67 s tests/signatures/test_adapter_image.py::test_image_input_formats[dspy_image_download-dspy.Image with download=True] PASSED [ 79%]
  1098. 1752.51 s tests/signatures/test_adapter_image.py::test_image_input_formats[dspy_image_no_download-dspy.Image without download] PASSED [ 79%]
  1099. 1754.49 s tests/signatures/test_adapter_image.py::test_predictor_save_load PASSED [ 79%]
  1100. 1756.55 s tests/signatures/test_adapter_image.py::test_save_load_complex_default_types PASSED [ 79%]
  1101. 1758.49 s tests/signatures/test_adapter_image.py::test_save_load_complex_types[test_case0] PASSED [ 79%]
  1102. 1760.48 s tests/signatures/test_adapter_image.py::test_save_load_complex_types[test_case1] PASSED [ 79%]
  1103. 1762.80 s tests/signatures/test_adapter_image.py::test_save_load_pydantic_model PASSED [ 79%]
  1104. 1764.99 s tests/signatures/test_adapter_image.py::test_optional_image_field PASSED [ 80%]
  1105. 1767.18 s tests/signatures/test_adapter_image.py::test_pdf_url_support PASSED [ 80%]
  1106. 1769.07 s tests/signatures/test_adapter_image.py::test_different_mime_types PASSED [ 80%]
  1107. 1770.84 s tests/signatures/test_adapter_image.py::test_mime_type_from_response_headers PASSED [ 80%]
  1108. 1772.65 s tests/signatures/test_adapter_image.py::test_pdf_from_file PASSED [ 80%]
  1109. 1774.94 s tests/signatures/test_adapter_image.py::test_image_repr PASSED [ 80%]
  1110. 1777.34 s tests/signatures/test_adapter_image.py::test_from_methods_warn PASSED [ 80%]
  1111. 1779.49 s tests/signatures/test_adapter_image.py::test_invalid_string_format PASSED [ 80%]
  1112. 1781.78 s tests/signatures/test_adapter_image.py::test_pil_image_with_download_parameter PASSED [ 80%]
  1113. 1783.84 s tests/signatures/test_custom_types.py::test_basic_custom_type_resolution PASSED [ 80%]
  1114. 1785.94 s tests/signatures/test_custom_types.py::test_type_alias_for_nested_types PASSED [ 80%]
  1115. 1788.05 s tests/signatures/test_custom_types.py::test_module_level_type_resolution PASSED [ 80%]
  1116. 1790.03 s tests/signatures/test_custom_types.py::test_recommended_patterns PASSED [ 80%]
  1117. 1792.18 s tests/signatures/test_custom_types.py::test_expected_failure PASSED [ 81%]
  1118. 1794.25 s tests/signatures/test_custom_types.py::test_module_type_resolution PASSED [ 81%]
  1119. 1796.28 s tests/signatures/test_signature.py::test_field_types_and_custom_attributes PASSED [ 81%]
  1120. 1798.28 s tests/signatures/test_signature.py::test_no_input_output PASSED [ 81%]
  1121. 1799.81 s tests/signatures/test_signature.py::test_no_input_output2 PASSED [ 81%]
  1122. 1801.15 s tests/signatures/test_signature.py::test_all_fields_have_prefix PASSED [ 81%]
  1123. 1802.41 s tests/signatures/test_signature.py::test_signature_parsing PASSED [ 81%]
  1124. 1804.30 s tests/signatures/test_signature.py::test_duplicate_input_output_field_names_raise PASSED [ 81%]
  1125. 1806.39 s tests/signatures/test_signature.py::test_with_signature PASSED [ 81%]
  1126. 1808.35 s tests/signatures/test_signature.py::test_append_instructions PASSED [ 81%]
  1127. 1810.28 s tests/signatures/test_signature.py::test_append_instructions_chains PASSED [ 81%]
  1128. 1812.19 s tests/signatures/test_signature.py::test_append_instructions_preserves_default_instructions PASSED [ 81%]
  1129. 1814.26 s tests/signatures/test_signature.py::test_with_updated_field PASSED [ 81%]
  1130. 1816.21 s tests/signatures/test_signature.py::test_empty_signature PASSED [ 81%]
  1131. 1818.27 s tests/signatures/test_signature.py::test_instructions_signature PASSED [ 82%]
  1132. 1820.19 s tests/signatures/test_signature.py::test_signature_instructions PASSED [ 82%]
  1133. 1822.25 s tests/signatures/test_signature.py::test_signature_instructions_none PASSED [ 82%]
  1134. 1824.38 s tests/signatures/test_signature.py::test_signature_from_dict PASSED [ 82%]
  1135. 1826.55 s tests/signatures/test_signature.py::test_signature_equality PASSED [ 82%]
  1136. 1828.58 s tests/signatures/test_signature.py::test_signature_inequality PASSED [ 82%]
  1137. 1831.00 s tests/signatures/test_signature.py::test_equality_format PASSED [ 82%]
  1138. 1832.93 s tests/signatures/test_signature.py::test_signature_reverse PASSED [ 82%]
  1139. 1835.02 s tests/signatures/test_signature.py::test_insert_field_at_various_positions PASSED [ 82%]
  1140. 1837.03 s tests/signatures/test_signature.py::test_order_preserved_with_mixed_annotations PASSED [ 82%]
  1141. 1839.10 s tests/signatures/test_signature.py::test_infer_prefix PASSED [ 82%]
  1142. 1841.07 s tests/signatures/test_signature.py::test_insantiating PASSED [ 82%]
  1143. 1843.27 s tests/signatures/test_signature.py::test_insantiating2 PASSED [ 82%]
  1144. 1845.28 s tests/signatures/test_signature.py::test_multiline_instructions PASSED [ 83%]
  1145. 1847.37 s tests/signatures/test_signature.py::test_dump_and_load_state PASSED [ 83%]
  1146. 1849.48 s tests/signatures/test_signature.py::test_typed_signatures_basic_types PASSED [ 83%]
  1147. 1851.54 s tests/signatures/test_signature.py::test_typed_signatures_generics PASSED [ 83%]
  1148. 1853.66 s tests/signatures/test_signature.py::test_typed_signatures_unions_and_optionals PASSED [ 83%]
  1149. 1855.84 s tests/signatures/test_signature.py::test_typed_signatures_any PASSED [ 83%]
  1150. 1857.76 s tests/signatures/test_signature.py::test_typed_signatures_nested PASSED [ 83%]
  1151. 1859.83 s tests/signatures/test_signature.py::test_typed_signatures_from_dict PASSED [ 83%]
  1152. 1861.98 s tests/signatures/test_signature.py::test_typed_signatures_complex_combinations PASSED [ 83%]
  1153. 1864.14 s tests/signatures/test_signature.py::test_make_signature_from_string PASSED [ 83%]
  1154. 1866.31 s tests/signatures/test_signature.py::test_signature_field_with_constraints PASSED [ 83%]
  1155. 1868.33 s tests/signatures/test_signature.py::test_basic_custom_type PASSED [ 83%]
  1156. 1870.53 s tests/signatures/test_signature.py::test_custom_type_from_different_module PASSED [ 83%]
  1157. 1873.00 s tests/signatures/test_signature.py::test_pep604_union_type_inline PASSED [ 84%]
  1158. 1875.50 s tests/signatures/test_signature.py::test_pep604_union_type_inline_equivalence PASSED [ 84%]
  1159. 1877.57 s tests/signatures/test_signature.py::test_pep604_union_type_inline_nested PASSED [ 84%]
  1160. 1879.59 s tests/signatures/test_signature.py::test_pep604_union_type_class_nested PASSED [ 84%]
  1161. 1881.74 s tests/signatures/test_signature.py::test_pep604_union_type_class_equivalence PASSED [ 84%]
  1162. 1883.82 s tests/signatures/test_signature.py::test_pep604_union_type_insert PASSED [ 84%]
  1163. 1885.87 s tests/signatures/test_signature.py::test_pep604_union_type_with_custom_types PASSED [ 84%]
  1164. 1888.12 s tests/signatures/test_signature.py::test_signature_cloudpickle_roundtrip PASSED [ 84%]
  1165. 1890.08 s tests/signatures/test_signature.py::test_predict_cloudpickle_roundtrip PASSED [ 84%]
  1166. 1894.97 s tests/streaming/test_streaming.py::test_streamify_yields_expected_response_chunks PASSED [ 84%]
  1167. 1899.65 s tests/streaming/test_streaming.py::test_streaming_response_yields_expected_response_chunks PASSED [ 84%]
  1168. 1901.94 s tests/streaming/test_streaming.py::test_default_status_streaming PASSED [ 84%]
  1169. 1903.90 s tests/streaming/test_streaming.py::test_custom_status_streaming PASSED [ 84%]
  1170. 1905.94 s tests/streaming/test_streaming.py::test_concurrent_status_message_providers PASSED [ 85%]
  1171. 1905.94 s tests/streaming/test_streaming.py::test_stream_listener_chat_adapter SKIPPED [ 85%]
  1172. 1907.98 s tests/streaming/test_streaming.py::test_default_status_streaming_in_async_program PASSED [ 85%]
  1173. 1908.00 s tests/streaming/test_streaming.py::test_stream_listener_json_adapter SKIPPED [ 85%]
  1174. 1910.24 s tests/streaming/test_streaming.py::test_streaming_handles_space_correctly PASSED [ 85%]
  1175. 1910.24 s tests/streaming/test_streaming.py::test_sync_streaming SKIPPED (need
  1176. 1910.25 s --llm_call option to run) [ 85%]
  1177. 1912.16 s tests/streaming/test_streaming.py::test_sync_status_streaming PASSED [ 85%]
  1178. 1913.89 s tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_chat_adapter PASSED [ 85%]
  1179. 1915.47 s tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_json_adapter PASSED [ 85%]
  1180. 1917.66 s tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_chat_adapter_untokenized_stream PASSED [ 85%]
  1181. 1919.72 s tests/streaming/test_streaming.py::test_stream_listener_missing_completion_marker_chat_adapter PASSED [ 85%]
  1182. 1921.83 s tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_json_adapter_untokenized_stream PASSED [ 85%]
  1183. 1925.11 s tests/streaming/test_streaming.py::test_status_message_non_blocking PASSED [ 85%]
  1184. 1928.89 s tests/streaming/test_streaming.py::test_status_message_non_blocking_async_program PASSED [ 86%]
  1185. 1931.09 s tests/streaming/test_streaming.py::test_stream_listener_allow_reuse PASSED [ 86%]
  1186. 1933.31 s tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_xml_adapter PASSED [ 86%]
  1187. 1935.46 s tests/streaming/test_streaming.py::test_streaming_allows_custom_chunk_types PASSED [ 86%]
  1188. 1938.27 s tests/streaming/test_streaming.py::test_streaming_allows_custom_streamable_type PASSED [ 86%]
  1189. 1941.43 s tests/streaming/test_streaming.py::test_streaming_with_citations PASSED [ 86%]
  1190. 1945.07 s tests/streaming/test_streaming.py::test_chat_adapter_simple_pydantic_streaming PASSED [ 86%]
  1191. 1948.86 s tests/streaming/test_streaming.py::test_chat_adapter_with_generic_type_annotation PASSED [ 86%]
  1192. 1952.62 s tests/streaming/test_streaming.py::test_chat_adapter_nested_pydantic_streaming PASSED [ 86%]
  1193. 1955.60 s tests/streaming/test_streaming.py::test_chat_adapter_mixed_fields_streaming PASSED [ 86%]
  1194. 1958.16 s tests/streaming/test_streaming.py::test_json_adapter_simple_pydantic_streaming PASSED [ 86%]
  1195. 1960.70 s tests/streaming/test_streaming.py::test_json_adapter_bracket_balance_detection PASSED [ 86%]
  1196. 1963.42 s tests/streaming/test_streaming.py::test_json_adapter_multiple_fields_detection PASSED [ 86%]
  1197. 1966.19 s tests/streaming/test_streaming.py::test_stream_listener_could_form_end_identifier_chat_adapter PASSED [ 86%]
  1198. 1968.62 s tests/streaming/test_streaming.py::test_stream_listener_could_form_end_identifier_json_adapter PASSED [ 87%]
  1199. 1971.59 s tests/streaming/test_streaming.py::test_stream_listener_could_form_end_identifier_xml_adapter PASSED [ 87%]
  1200. 1974.16 s tests/streaming/test_streaming.py::test_streaming_reasoning_model PASSED [ 87%]
  1201. 1977.24 s tests/streaming/test_streaming.py::test_stream_listener_empty_last_chunk_chat_adapter PASSED [ 87%]
  1202. 1980.46 s tests/streaming/test_streaming.py::test_stream_listener_empty_last_chunk_json_adapter PASSED [ 87%]
  1203. 1983.59 s tests/streaming/test_streaming.py::test_streaming_reasoning_fallback PASSED [ 87%]
  1204. 1986.66 s tests/teleprompt/test_bettertogether.py::test_bettertogether_import PASSED [ 87%]
  1205. 1989.84 s tests/teleprompt/test_bettertogether.py::test_bettertogether_initialization_default PASSED [ 87%]
  1206. 1992.61 s tests/teleprompt/test_bettertogether.py::test_bettertogether_initialization_custom PASSED [ 87%]
  1207. 1995.77 s tests/teleprompt/test_bettertogether.py::test_bettertogether_initialization_invalid_optimizer PASSED [ 87%]
  1208. 1999.86 s tests/teleprompt/test_bettertogether.py::test_strategy_validation PASSED [ 87%]
  1209. 2003.51 s tests/teleprompt/test_bettertogether.py::test_compile_basic PASSED [ 87%]
  1210. 2007.52 s tests/teleprompt/test_bettertogether.py::test_trainset_validation PASSED [ 87%]
  1211. 2011.45 s tests/teleprompt/test_bettertogether.py::test_valset_ratio_validation PASSED [ 88%]
  1212. 2015.43 s tests/teleprompt/test_bettertogether.py::test_optimizer_compile_args_validation PASSED [ 88%]
  1213. 2019.27 s tests/teleprompt/test_bettertogether.py::test_student_in_optimizer_compile_args PASSED [ 88%]
  1214. 2023.23 s tests/teleprompt/test_bettertogether.py::test_compile_args_passed_to_optimizer PASSED [ 88%]
  1215. 2026.94 s tests/teleprompt/test_bettertogether.py::test_compile_args_multi_optimizer_strategy PASSED [ 88%]
  1216. 2031.02 s tests/teleprompt/test_bettertogether.py::test_compile_args_override_global_params PASSED [ 88%]
  1217. 2034.90 s tests/teleprompt/test_bettertogether.py::test_trainset_shuffling_between_steps PASSED [ 88%]
  1218. 2038.54 s tests/teleprompt/test_bettertogether.py::test_strategy_execution_order PASSED [ 88%]
  1219. 2042.37 s tests/teleprompt/test_bettertogether.py::test_lm_lifecycle_management PASSED [ 88%]
  1220. 2046.19 s tests/teleprompt/test_bettertogether.py::test_error_handling_returns_best_program PASSED [ 88%]
  1221. 2049.80 s tests/teleprompt/test_bettertogether.py::test_program_selection[test_valset0-p_optimized-With valset: returns best score (p), not latest (w)] PASSED [ 88%]
  1222. 2053.48 s tests/teleprompt/test_bettertogether.py::test_program_selection[None-w_optimized-Without valset: returns latest program (w)] PASSED [ 88%]
  1223. 2057.35 s tests/teleprompt/test_bettertogether.py::test_candidate_programs_structure PASSED [ 88%]
  1224. 2061.28 s tests/teleprompt/test_bettertogether.py::test_empty_valset_handling PASSED [ 89%]
  1225. 2065.05 s tests/teleprompt/test_bootstrap.py::test_bootstrap_initialization PASSED [ 89%]
  1226. 2068.74 s tests/teleprompt/test_bootstrap.py::test_compile_with_predict_instances PASSED [ 89%]
  1227. 2072.60 s tests/teleprompt/test_bootstrap.py::test_bootstrap_effectiveness PASSED [ 89%]
  1228. 2076.57 s tests/teleprompt/test_bootstrap.py::test_error_handling_during_bootstrap PASSED [ 89%]
  1229. 2080.41 s tests/teleprompt/test_bootstrap.py::test_validation_set_usage PASSED [ 89%]
  1230. 2083.90 s tests/teleprompt/test_bootstrap_finetune.py::test_bootstrap_finetune_initialization PASSED [ 89%]
  1231. 2087.22 s tests/teleprompt/test_bootstrap_finetune.py::test_compile_with_predict_instances PASSED [ 89%]
  1232. 2090.06 s tests/teleprompt/test_bootstrap_finetune.py::test_error_handling_missing_lm PASSED [ 89%]
  1233. 2092.37 s tests/teleprompt/test_bootstrap_trace.py::test_bootstrap_trace_data PASSED [ 89%]
  1234. 2094.94 s tests/teleprompt/test_bootstrap_trace.py::test_bootstrap_trace_data_passes_callback_metadata PASSED [ 89%]
  1235. 2097.41 s tests/teleprompt/test_copro_optimizer.py::test_signature_optimizer_initialization PASSED [ 89%]
  1236. 2099.94 s tests/teleprompt/test_copro_optimizer.py::test_signature_optimizer_optimization_process PASSED [ 89%]
  1237. 2102.31 s tests/teleprompt/test_copro_optimizer.py::test_signature_optimizer_statistics_tracking PASSED [ 90%]
  1238. 2104.89 s tests/teleprompt/test_copro_optimizer.py::test_optimization_and_output_verification PASSED [ 90%]
  1239. 2107.15 s tests/teleprompt/test_copro_optimizer.py::test_statistics_tracking_during_optimization PASSED [ 90%]
  1240. 2109.23 s tests/teleprompt/test_ensemble.py::test_ensemble_without_reduction PASSED [ 90%]
  1241. 2111.16 s tests/teleprompt/test_ensemble.py::test_ensemble_with_reduction PASSED [ 90%]
  1242. 2113.33 s tests/teleprompt/test_ensemble.py::test_ensemble_with_size_limitation PASSED [ 90%]
  1243. 2115.31 s tests/teleprompt/test_ensemble.py::test_ensemble_deterministic_behavior PASSED [ 90%]
  1244. 2117.42 s tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[None-batch0-expected_callback_metadata0] PASSED [ 90%]
  1245. 2119.52 s tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[None-batch1-expected_callback_metadata1] PASSED [ 90%]
  1246. 2121.65 s tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[1-batch2-expected_callback_metadata2] PASSED [ 90%]
  1247. 2123.58 s tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[1-batch3-expected_callback_metadata3] PASSED [ 90%]
  1248. 2125.45 s tests/teleprompt/test_gepa.py::test_basic_workflow[True] PASSED [ 90%]
  1249. 2127.46 s tests/teleprompt/test_gepa.py::test_basic_workflow[False] PASSED [ 90%]
  1250. 2129.56 s tests/teleprompt/test_gepa.py::test_workflow_with_custom_instruction_proposer_and_component_selector PASSED [ 90%]
  1251. 2131.76 s tests/teleprompt/test_gepa.py::test_metric_requires_feedback_signature PASSED [ 91%]
  1252. 2133.80 s tests/teleprompt/test_gepa.py::test_reflection_prompt_template_in_gepa_kwargs_raises PASSED [ 91%]
  1253. 2135.83 s tests/teleprompt/test_gepa.py::test_gepa_compile_with_track_usage_no_tuple_error PASSED [ 91%]
  1254. 2137.78 s tests/teleprompt/test_gepa.py::test_component_selector_functionality PASSED [ 91%]
  1255. 2139.80 s tests/teleprompt/test_gepa.py::test_component_selector_default_behavior PASSED [ 91%]
  1256. 2141.83 s tests/teleprompt/test_gepa.py::test_component_selector_string_round_robin PASSED [ 91%]
  1257. 2143.85 s tests/teleprompt/test_gepa.py::test_component_selector_string_all PASSED [ 91%]
  1258. 2145.81 s tests/teleprompt/test_gepa.py::test_component_selector_custom_random PASSED [ 91%]
  1259. 2147.66 s tests/teleprompt/test_gepa.py::test_alternating_half_component_selector PASSED [ 91%]
  1260. 2149.64 s tests/teleprompt/test_gepa.py::test_track_stats_result_structure PASSED [ 91%]
  1261. 2151.35 s tests/teleprompt/test_gepa.py::test_track_best_outputs_result_structure PASSED [ 91%]
  1262. 2153.08 s tests/teleprompt/test_gepa_instruction_proposer.py::test_reflection_lm_gets_structured_images PASSED [ 91%]
  1263. 2154.84 s tests/teleprompt/test_gepa_instruction_proposer.py::test_custom_proposer_without_reflection_lm PASSED [ 91%]
  1264. 2156.68 s tests/teleprompt/test_gepa_instruction_proposer.py::test_image_serialization_into_strings PASSED [ 92%]
  1265. 2158.60 s tests/teleprompt/test_gepa_instruction_proposer.py::test_default_proposer[True] PASSED [ 92%]
  1266. 2160.48 s tests/teleprompt/test_gepa_instruction_proposer.py::test_default_proposer[False] PASSED [ 92%]
  1267. 2162.20 s tests/teleprompt/test_grpo.py::test_grpo_dataset_shuffler PASSED [ 92%]
  1268. 2164.19 s tests/teleprompt/test_grpo.py::test_grpo_dataset_shuffler_with_num_ex_per_step_less_dataset PASSED [ 92%]
  1269. 2166.19 s tests/teleprompt/test_grpo.py::test_grpo_dataset_shuffler_with_num_ex_per_step_greater_dataset PASSED [ 92%]
  1270. 2168.16 s tests/teleprompt/test_knn_fewshot.py::test_knn_few_shot_initialization PASSED [ 92%]
  1271. 2170.57 s tests/teleprompt/test_random_search.py::test_basic_workflow PASSED [ 92%]
  1272. 2172.69 s tests/teleprompt/test_teleprompt.py::test_get_params PASSED [ 92%]
  1273. 2174.71 s tests/teleprompt/test_utils.py::test_eval_candidate_program_full_trainset PASSED [ 92%]
  1274. 2176.53 s tests/teleprompt/test_utils.py::test_eval_candidate_program_minibatch PASSED [ 92%]
  1275. 2178.48 s tests/teleprompt/test_utils.py::test_eval_candidate_program_failure PASSED [ 92%]
  1276. 2180.28 s tests/teleprompt/test_utils.py::test_create_n_fewshot_demo_sets_passes_metric_threshold_for_unshuffled PASSED [ 92%]
  1277. 2182.19 s tests/utils/test_annotation.py::test_experimental_decorator_on_function PASSED [ 93%]
  1278. 2183.96 s tests/utils/test_annotation.py::test_experimental_decorator_on_function_with_version PASSED [ 93%]
  1279. 2186.01 s tests/utils/test_annotation.py::test_experimental_decorator_on_class PASSED [ 93%]
  1280. 2187.75 s tests/utils/test_annotation.py::test_experimental_decorator_on_class_with_version PASSED [ 93%]
  1281. 2189.50 s tests/utils/test_annotation.py::test_experimental_decorator_without_docstring PASSED [ 93%]
  1282. 2191.20 s tests/utils/test_annotation.py::test_experimental_decorator_without_docstring_with_version PASSED [ 93%]
  1283. 2193.03 s tests/utils/test_annotation.py::test_experimental_decorator_with_callable_syntax PASSED [ 93%]
  1284. 2194.86 s tests/utils/test_annotation.py::test_experimental_decorator_with_version_callable_syntax PASSED [ 93%]
  1285. 2196.81 s tests/utils/test_asyncify.py::test_async_limiter PASSED [ 93%]
  1286. 2201.81 s tests/utils/test_asyncify.py::test_asyncify
  1287. 2203.09 s PASSED [ 93%]
  1288. 2205.28 s tests/utils/test_exceptions.py::test_lm_errors_are_exported_from_dspy PASSED [ 93%]
  1289. 2207.24 s tests/utils/test_exceptions.py::test_retryable_lm_errors_classification PASSED [ 93%]
  1290. 2209.71 s tests/utils/test_exceptions.py::test_lm_error_metadata PASSED [ 93%]
  1291. 2211.76 s tests/utils/test_exceptions.py::test_context_window_exceeded_error_defaults PASSED [ 94%]
  1292. 2214.19 s tests/utils/test_exceptions.py::test_context_window_exceeded_error_with_model PASSED [ 94%]
  1293. 2216.42 s tests/utils/test_exceptions.py::test_context_window_exceeded_error_with_message PASSED [ 94%]
  1294. 2218.33 s tests/utils/test_exceptions.py::test_context_window_exceeded_error_message_without_model PASSED [ 94%]
  1295. 2220.63 s tests/utils/test_exceptions.py::test_adapter_parse_error_basic PASSED [ 94%]
  1296. 2222.72 s tests/utils/test_exceptions.py::test_adapter_parse_error_with_message PASSED [ 94%]
  1297. 2224.64 s tests/utils/test_exceptions.py::test_adapter_parse_error_with_parsed_result PASSED [ 94%]
  1298. 2226.78 s tests/utils/test_lazy_import.py::test_is_available_true_for_stdlib PASSED [ 94%]
  1299. 2228.50 s tests/utils/test_lazy_import.py::test_is_available_false_for_missing PASSED [ 94%]
  1300. 2230.35 s tests/utils/test_lazy_import.py::test_is_available_does_not_import_module PASSED [ 94%]
  1301. 2232.08 s tests/utils/test_lazy_import.py::test_require_returns_lazy_module_when_present PASSED [ 94%]
  1302. 2233.81 s tests/utils/test_lazy_import.py::test_require_returns_cached_module PASSED [ 94%]
  1303. 2235.72 s tests/utils/test_lazy_import.py::test_require_is_safe_under_concurrent_first_use PASSED [ 94%]
  1304. 2237.66 s tests/utils/test_lazy_import.py::test_require_assignment_updates_materialized_module PASSED [ 95%]
  1305. 2239.59 s tests/utils/test_lazy_import.py::test_require_returns_stub_when_missing PASSED [ 95%]
  1306. 2241.51 s tests/utils/test_lazy_import.py::test_require_stub_raises_on_access_with_install_hint PASSED [ 95%]
  1307. 2243.49 s tests/utils/test_lazy_import.py::test_require_stub_uses_install_hint_for_litellm PASSED [ 95%]
  1308. 2245.34 s tests/utils/test_lazy_import.py::test_require_stub_uses_explicit_extra PASSED [ 95%]
  1309. 2247.22 s tests/utils/test_lazy_import.py::test_require_stub_falls_back_to_module_name PASSED [ 95%]
  1310. 2249.14 s tests/utils/test_lazy_import.py::test_install_hints_match_pyproject_extras PASSED [ 95%]
  1311. 2251.17 s tests/utils/test_magicattr.py::test_magicattr_get[l-value0] PASSED [ 95%]
  1312. 2253.30 s tests/utils/test_magicattr.py::test_magicattr_get[t.t.t.t.z-z] PASSED [ 95%]
  1313. 2255.12 s tests/utils/test_magicattr.py::test_magicattr_get[a[0]-0] PASSED [ 95%]
  1314. 2257.07 s tests/utils/test_magicattr.py::test_magicattr_get[a[1][0]-1] PASSED [ 95%]
  1315. 2259.18 s tests/utils/test_magicattr.py::test_magicattr_get[a[1][2]-value4] PASSED [ 95%]
  1316. 2261.12 s tests/utils/test_magicattr.py::test_magicattr_get[b["x"]-value5] PASSED [ 95%]
  1317. 2262.96 s tests/utils/test_magicattr.py::test_magicattr_get[b["x"]["y"]-y] PASSED [ 95%]
  1318. 2265.09 s tests/utils/test_magicattr.py::test_magicattr_get[b["z"]-value7] PASSED [ 96%]
  1319. 2267.29 s tests/utils/test_magicattr.py::test_magicattr_get[b["z"][1]-2] PASSED [ 96%]
  1320. 2269.18 s tests/utils/test_magicattr.py::test_magicattr_get[b["w"].z-z] PASSED [ 96%]
  1321. 2271.18 s tests/utils/test_magicattr.py::test_magicattr_get[b["w"].t.l-value10] PASSED [ 96%]
  1322. 2273.36 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z] PASSED [ 96%]
  1323. 2275.34 s tests/utils/test_magicattr.py::test_magicattr_get[l[-1]-2] PASSED [ 96%]
  1324. 2277.41 s tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.a[-1].z-z] PASSED [ 96%]
  1325. 2279.70 s tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.b["z"][0]-1] PASSED [ 96%]
  1326. 2281.86 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].t.z-z] PASSED [ 96%]
  1327. 2284.02 s tests/utils/test_magicattr.py::test_person_example PASSED [ 96%]
  1328. 2286.49 s tests/utils/test_magicattr.py::test_empty PASSED [ 96%]
  1329. 2286.51 s tests/utils/test_mcp.py::test_convert_mcp_tool SKIPPED (need --extra
  1330. 2286.51 s option to run) [ 96%]
  1331. 2289.30 s tests/utils/test_parallelizer.py::test_worker_threads_independence PASSED [ 96%]
  1332. 2291.57 s tests/utils/test_parallelizer.py::test_parallel_execution_speed PASSED [ 97%]
  1333. 2293.92 s tests/utils/test_parallelizer.py::test_max_errors_handling PASSED [ 97%]
  1334. 2295.97 s tests/utils/test_parallelizer.py::test_max_errors_not_met PASSED [ 97%]
  1335. 2298.04 s tests/utils/test_parallelizer.py::test_parallel_executor_tracks_failed_indices_and_exceptions PASSED [ 97%]
  1336. 2300.60 s tests/utils/test_parallelizer.py::test_sequential_execution_runs_on_main_thread PASSED [ 97%]
  1337. 2302.79 s tests/utils/test_parallelizer.py::test_sequential_max_errors_not_met PASSED [ 97%]
  1338. 2305.25 s tests/utils/test_parallelizer.py::test_sequential_max_errors_exceeded PASSED [ 97%]
  1339. 2307.68 s tests/utils/test_parallelizer.py::test_sequential_tracks_failed_indices_and_exceptions PASSED [ 97%]
  1340. 2310.23 s tests/utils/test_parallelizer.py::test_sequential_compare_results PASSED [ 97%]
  1341. 2312.60 s tests/utils/test_saving.py::test_save_predict PASSED [ 97%]
  1342. 2315.20 s tests/utils/test_saving.py::test_save_custom_model PASSED [ 97%]
  1343. 2317.85 s tests/utils/test_saving.py::test_save_model_with_custom_signature PASSED [ 97%]
  1344. 2317.87 s tests/utils/test_saving.py::test_save_compiled_model SKIPPED (need
  1345. 2317.87 s --extra option to run) [ 97%]
  1346. 2320.47 s tests/utils/test_saving.py::test_load_with_version_mismatch PASSED [ 98%]
  1347. 2322.78 s tests/utils/test_saving.py::test_pickle_loading_requires_explicit_permission PASSED [ 98%]
  1348. 2324.93 s tests/utils/test_saving.py::test_pkl_file_loading_requires_explicit_permission PASSED [ 98%]
  1349. 2327.18 s tests/utils/test_saving.py::test_json_file_loading_works_without_permission PASSED [ 98%]
  1350. 2329.47 s tests/utils/test_settings.py::test_basic_dspy_settings PASSED [ 98%]
  1351. 2331.67 s tests/utils/test_settings.py::test_forbid_configure_call_in_child_thread PASSED [ 98%]
  1352. 2333.76 s tests/utils/test_settings.py::test_dspy_context PASSED [ 98%]
  1353. 2335.94 s tests/utils/test_settings.py::test_dspy_context_parallel PASSED [ 98%]
  1354. 2338.97 s tests/utils/test_settings.py::test_dspy_context_with_dspy_parallel PASSED [ 98%]
  1355. 2342.39 s tests/utils/test_settings.py::test_dspy_context_with_async_task_group PASSED [ 98%]
  1356. 2344.93 s tests/utils/test_settings.py::test_dspy_configure_allowance_async PASSED [ 98%]
  1357. 2346.97 s tests/utils/test_settings.py::test_dspy_settings_save_load PASSED [ 98%]
  1358. 2349.16 s tests/utils/test_settings.py::test_dspy_settings_save_exclude_keys PASSED [ 98%]
  1359. 2351.39 s tests/utils/test_settings.py::test_settings_save_with_extra_modules PASSED [ 99%]
  1360. 2353.49 s tests/utils/test_syncify.py::test_syncify_in_place PASSED [ 99%]
  1361. 2355.68 s tests/utils/test_syncify.py::test_syncify_with_wrapper PASSED [ 99%]
  1362. 2357.78 s tests/utils/test_syncify.py::test_syncify_works_with_optimizers PASSED [ 99%]
  1363. 2362.81 s tests/utils/test_unbatchify.py::test_unbatchify_batch_size_trigger
  1364. 2364.96 s PASSED [ 99%]
  1365. 2367.58 s tests/utils/test_unbatchify.py::test_unbatchify_timeout_trigger PASSED [ 99%]
  1366. 2369.83 s tests/utils/test_usage_tracker.py::test_add_usage_entry PASSED [ 99%]
  1367. 2371.76 s tests/utils/test_usage_tracker.py::test_get_total_tokens PASSED [ 99%]
  1368. 2373.84 s tests/utils/test_usage_tracker.py::test_track_usage_with_multiple_models PASSED [ 99%]
  1369. 2375.95 s tests/utils/test_usage_tracker.py::test_track_usage_context_manager SKIPPED [ 99%]
  1370. 2377.88 s tests/utils/test_usage_tracker.py::test_merge_usage_entries_with_new_keys PASSED [ 99%]
  1371. 2380.05 s tests/utils/test_usage_tracker.py::test_merge_usage_entries_with_none_values PASSED [ 99%]
  1372. 2382.13 s tests/utils/test_usage_tracker.py::test_merge_usage_entries_with_pydantic_models PASSED [ 99%]
  1373. 2384.03 s tests/utils/test_usage_tracker.py::test_parallel_executor_with_usage_tracker PASSED [100%]
  1374. 2384.03 s
  1375. 2384.03 s =============================== warnings summary ===============================
  1376. 2384.03 s tests/clients/test_lm.py::test_chat_lms_can_be_queried
  1377. 2384.03 s tests/clients/test_lm.py::test_lm_calls_support_callables
  1378. 2384.03 s tests/clients/test_lm.py::test_lm_calls_support_pydantic_models
  1379. 2384.03 s /nix/store/rn0r480kscbk8159qi3qjflil71kqglq-dspy-test-env/lib/python3.13/site-packages/pydantic/main.py:464: UserWarning: Pydantic serializer warnings:
  1380. 2384.03 s PydanticSerializationUnexpectedValue(Expected 9 fields but got 5: Expected `Message` - serialized value may not be as expected [field_name='choices', input_value=Message(content='Hi!', ro...ields={'refusal': None}), input_type=Message])
  1381. 2384.03 s PydanticSerializationUnexpectedValue(Expected `StreamingChoices` - serialized value may not be as expected [field_name='choices', input_value=Choices(finish_reason='st...elds={'refusal': None})), input_type=Choices])
  1382. 2384.03 s return self.__pydantic_serializer__.to_python(
  1383. 2384.03 s
  1384. 2384.03 s tests/predict/test_multi_chain_comparison.py::test_basic_example
  1385. 2384.03 s tests/predict/test_multi_chain_comparison.py::test_basic_example
  1386. 2384.03 s tests/predict/test_multi_chain_comparison.py::test_basic_example
  1387. 2384.03 s /tmp/nix-ci-worker/stable-pwd/repo/dspy/predict/multi_chain_comparison.py:19: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version.
  1388. 2384.03 s InputField(
  1389. 2384.03 s
  1390. 2384.03 s tests/predict/test_multi_chain_comparison.py::test_basic_example
  1391. 2384.03 s /tmp/nix-ci-worker/stable-pwd/repo/dspy/predict/multi_chain_comparison.py:27: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version.
  1392. 2384.03 s OutputField(
  1393. 2384.03 s
  1394. 2384.03 s tests/predict/test_predict.py::test_signature_fields_after_dump_and_load_state
  1395. 2384.03 s /tmp/nix-ci-worker/stable-pwd/repo/tests/predict/test_predict.py:299: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version.
  1396. 2384.03 s sentiment = dspy.OutputField(desc="I am a malicious output!", prefix="I am a prefix!")
  1397. 2384.03 s
  1398. 2384.03 s tests/signatures/test_signature.py::test_all_fields_have_prefix
  1399. 2384.03 s /tmp/nix-ci-worker/stable-pwd/repo/tests/signatures/test_signature.py:46: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version.
  1400. 2384.03 s input = InputField(prefix="Modified:")
  1401. 2384.03 s
  1402. 2384.03 s tests/signatures/test_signature.py::test_equality_format
  1403. 2384.03 s /tmp/nix-ci-worker/stable-pwd/repo/tests/signatures/test_signature.py:157: DeprecationWarning: The 'format' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version.
  1404. 2384.03 s input = InputField(format=lambda x: x)
  1405. 2384.03 s
  1406. 2384.03 s tests/streaming/test_streaming.py::test_streamify_yields_expected_response_chunks
  1407. 2384.03 s tests/streaming/test_streaming.py::test_streaming_response_yields_expected_response_chunks
  1408. 2384.03 s /nix/store/rn0r480kscbk8159qi3qjflil71kqglq-dspy-test-env/lib/python3.13/site-packages/pydantic/main.py:464: UserWarning: Pydantic serializer warnings:
  1409. 2384.03 s PydanticSerializationUnexpectedValue(Expected 9 fields but got 5: Expected `Message` - serialized value may not be as expected [field_name='choices', input_value=Message(content='{"output...er_specific_fields=None), input_type=Message])
  1410. 2384.03 s PydanticSerializationUnexpectedValue(Expected `StreamingChoices` - serialized value may not be as expected [field_name='choices', input_value=Choices(finish_reason='st...r_specific_fields=None)), input_type=Choices])
  1411. 2384.03 s return self.__pydantic_serializer__.to_python(
  1412. 2384.03 s
  1413. 2384.03 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z]
  1414. 2384.03 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z]
  1415. 2384.03 s tests/utils/test_magicattr.py::test_magicattr_get[l[-1]-2]
  1416. 2384.03 s tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.a[-1].z-z]
  1417. 2384.03 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].t.z-z]
  1418. 2384.03 s tests/utils/test_magicattr.py::test_person_example
  1419. 2384.03 s tests/utils/test_magicattr.py::test_person_example
  1420. 2384.03 s tests/utils/test_magicattr.py::test_person_example
  1421. 2384.03 s tests/utils/test_magicattr.py::test_person_example
  1422. 2384.03 s /tmp/nix-ci-worker/stable-pwd/repo/dspy/utils/magicattr.py:99: DeprecationWarning: ast.Num is deprecated and will be removed in Python 3.14; use ast.Constant instead
  1423. 2384.03 s if hasattr(ast, "Num") and isinstance(node, ast.Num):
  1424. 2384.03 s
  1425. 2384.03 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z]
  1426. 2384.03 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z]
  1427. 2384.03 s tests/utils/test_magicattr.py::test_magicattr_get[l[-1]-2]
  1428. 2384.03 s tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.a[-1].z-z]
  1429. 2384.03 s tests/utils/test_magicattr.py::test_magicattr_get[a[-1].t.z-z]
  1430. 2384.03 s tests/utils/test_magicattr.py::test_person_example
  1431. 2384.03 s tests/utils/test_magicattr.py::test_person_example
  1432. 2384.03 s tests/utils/test_magicattr.py::test_person_example
  1433. 2384.03 s tests/utils/test_magicattr.py::test_person_example
  1434. 2384.03 s /tmp/nix-ci-worker/stable-pwd/repo/dspy/utils/magicattr.py:102: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
  1435. 2384.03 s elif hasattr(ast, "Str") and isinstance(node, ast.Str):
  1436. 2384.03 s
  1437. 2384.03 s tests/utils/test_settings.py::test_dspy_configure_allowance_async
  1438. 2384.03 s /tmp/nix-ci-worker/stable-pwd/repo/tests/utils/test_settings.py:194: RuntimeWarning: coroutine 'test_dspy_configure_allowance_async.<locals>.foo4' was never awaited
  1439. 2384.03 s foo4()
  1440. 2384.03 s Enable tracemalloc to get traceback where the object was allocated.
  1441. 2384.03 s See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
  1442. 2384.03 s
  1443. 2384.03 s -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
  1444. 2384.03 s ==== 991 passed, 330 skipped, 2 xfailed, 31 warnings in 2373.27s (0:39:33) =====
  1445. 2385.66 s Run succeeded.
More Logs Raw Logs

Agent Instructions

Wait for completion, without logs:

curl --netrc -N https://staging.nix-ci.com/gh:NorfairKing:dspy/your-first-flake/453ecf257a343742170b3e1cef20271508c3e6a4/d74fba83-b850-4219-8abd-855bb97e1167/wait

Latest logs, with timestamps (streams while running):

curl --netrc -N https://staging.nix-ci.com/gh:NorfairKing:dspy/your-first-flake/453ecf257a343742170b3e1cef20271508c3e6a4/d74fba83-b850-4219-8abd-855bb97e1167/tail

All logs, with timestamps (available after completion):

curl --netrc https://staging.nix-ci.com/gh:NorfairKing:dspy/your-first-flake/453ecf257a343742170b3e1cef20271508c3e6a4/d74fba83-b850-4219-8abd-855bb97e1167/logs

Raw logs, from the start (no timestamps):

curl --netrc -N https://staging.nix-ci.com/gh:NorfairKing:dspy/your-first-flake/453ecf257a343742170b3e1cef20271508c3e6a4/d74fba83-b850-4219-8abd-855bb97e1167/raw-logs

Content types available for /logs and /tail (use Accept header):

Example:

curl --netrc -H "Accept: text/csv" https://staging.nix-ci.com/gh:NorfairKing:dspy/your-first-flake/453ecf257a343742170b3e1cef20271508c3e6a4/d74fba83-b850-4219-8abd-855bb97e1167/logs

Reproduce locally: https://staging.nix-ci.com/gh:NorfairKing:dspy/your-first-flake/453ecf257a343742170b3e1cef20271508c3e6a4/d74fba83-b850-4219-8abd-855bb97e1167/repro

This page is also available in machine-friendly formats. Request one with the Accept header:

For more instructions, fetch https://staging.nix-ci.com/instructions .