$ /nix/store/9ypz3flqsrl5xl495mm8h645gadjsxi1-coreutils-9.11/bin/timeout --kill-after=15s 7200s /nix/store/23zk8sgzamrbmj1fday15szf7f2279s8-nix-2.34.7/bin/nix --extra-experimental-features nix-command --extra-experimental-features flakes flake clone git+https://github.com/NorfairKing/dspy --dest /tmp/nix-ci-worker/stable-pwd/repo/ Cloning into '/tmp/nix-ci-worker/stable-pwd/repo'... remote: Enumerating objects: 54719, done. remote: Counting objects: 1% (1/78) remote: Counting objects: 2% (2/78) remote: Counting objects: 3% (3/78) remote: Counting objects: 5% (4/78) remote: Counting objects: 6% (5/78) remote: Counting objects: 7% (6/78) remote: Counting objects: 8% (7/78) remote: Counting objects: 10% (8/78) remote: Counting objects: 11% (9/78) remote: Counting objects: 12% (10/78) remote: Counting objects: 14% (11/78) remote: Counting objects: 15% (12/78) remote: Counting objects: 16% (13/78) remote: Counting objects: 17% (14/78) remote: Counting objects: 19% (15/78) remote: Counting objects: 20% (16/78) remote: Counting objects: 21% (17/78) remote: Counting objects: 23% (18/78) remote: Counting objects: 24% (19/78) remote: Counting objects: 25% (20/78) remote: Counting objects: 26% (21/78) remote: Counting objects: 28% (22/78) remote: Counting objects: 29% (23/78) remote: Counting objects: 30% (24/78) remote: Counting objects: 32% (25/78) remote: Counting objects: 33% (26/78) remote: Counting objects: 34% (27/78) remote: Counting objects: 35% (28/78) remote: Counting objects: 37% (29/78) remote: Counting objects: 38% (30/78) remote: Counting objects: 39% (31/78) remote: Counting objects: 41% (32/78) remote: Counting objects: 42% (33/78) remote: Counting objects: 43% (34/78) remote: Counting objects: 44% (35/78) remote: Counting objects: 46% (36/78) remote: Counting objects: 47% (37/78) remote: Counting objects: 48% (38/78) remote: Counting objects: 50% (39/78) remote: Counting objects: 51% (40/78) remote: Counting objects: 52% (41/78) remote: Counting objects: 53% (42/78) remote: Counting objects: 55% (43/78) remote: Counting objects: 56% (44/78) remote: Counting objects: 57% (45/78) remote: Counting objects: 58% (46/78) remote: Counting objects: 60% (47/78) remote: Counting objects: 61% (48/78) remote: Counting objects: 62% (49/78) remote: Counting objects: 64% (50/78) remote: Counting objects: 65% (51/78) remote: Counting objects: 66% (52/78) remote: Counting objects: 67% (53/78) remote: Counting objects: 69% (54/78) remote: Counting objects: 70% (55/78) remote: Counting objects: 71% (56/78) remote: Counting objects: 73% (57/78) remote: Counting objects: 74% (58/78) remote: Counting objects: 75% (59/78) remote: Counting objects: 76% (60/78) remote: Counting objects: 78% (61/78) remote: Counting objects: 79% (62/78) remote: Counting objects: 80% (63/78) remote: Counting objects: 82% (64/78) remote: Counting objects: 83% (65/78) remote: Counting objects: 84% (66/78) remote: Counting objects: 85% (67/78) remote: Counting objects: 87% (68/78) remote: Counting objects: 88% (69/78) remote: Counting objects: 89% (70/78) remote: Counting objects: 91% (71/78) remote: Counting objects: 92% (72/78) remote: Counting objects: 93% (73/78) remote: Counting objects: 94% (74/78) remote: Counting objects: 96% (75/78) remote: Counting objects: 97% (76/78) remote: Counting objects: 98% (77/78) remote: Counting objects: 100% (78/78) remote: Counting objects: 100% (78/78), done. remote: Compressing objects: 2% (1/50) remote: Compressing objects: 4% (2/50) remote: Compressing objects: 6% (3/50) remote: Compressing objects: 8% (4/50) remote: Compressing objects: 10% (5/50) remote: Compressing objects: 12% (6/50) remote: Compressing objects: 14% (7/50) remote: Compressing objects: 16% (8/50) remote: Compressing objects: 18% (9/50) remote: Compressing objects: 20% (10/50) remote: Compressing objects: 22% (11/50) remote: Compressing objects: 24% (12/50) remote: Compressing objects: 26% (13/50) remote: Compressing objects: 28% (14/50) remote: Compressing objects: 30% (15/50) remote: Compressing objects: 32% (16/50) remote: Compressing objects: 34% (17/50) remote: Compressing objects: 36% (18/50) remote: Compressing objects: 38% (19/50) remote: Compressing objects: 40% (20/50) remote: Compressing objects: 42% (21/50) remote: Compressing objects: 44% (22/50) remote: Compressing objects: 46% (23/50) remote: Compressing objects: 48% (24/50) remote: Compressing objects: 50% (25/50) remote: Compressing objects: 52% (26/50) remote: Compressing objects: 54% (27/50) remote: Compressing objects: 56% (28/50) remote: Compressing objects: 58% (29/50) remote: Compressing objects: 60% (30/50) remote: Compressing objects: 62% (31/50) remote: Compressing objects: 64% (32/50) remote: Compressing objects: 66% (33/50) remote: Compressing objects: 68% (34/50) remote: Compressing objects: 70% (35/50) remote: Compressing objects: 72% (36/50) remote: Compressing objects: 74% (37/50) remote: Compressing objects: 76% (38/50) remote: Compressing objects: 78% (39/50) remote: Compressing objects: 80% (40/50) remote: Compressing objects: 82% (41/50) remote: Compressing objects: 84% (42/50) remote: Compressing objects: 86% (43/50) remote: Compressing objects: 88% (44/50) remote: Compressing objects: 90% (45/50) remote: Compressing objects: 92% (46/50) remote: Compressing objects: 94% (47/50) remote: Compressing objects: 96% (48/50) remote: Compressing objects: 98% (49/50) remote: Compressing objects: 100% (50/50) remote: Compressing objects: 100% (50/50), done. Receiving objects: 0% (1/54719) Receiving objects: 1% (548/54719) Receiving objects: 2% (1095/54719) Receiving objects: 3% (1642/54719) Receiving objects: 4% (2189/54719) Receiving objects: 5% (2736/54719) Receiving objects: 6% (3284/54719) Receiving objects: 7% (3831/54719) Receiving objects: 8% (4378/54719) Receiving objects: 9% (4925/54719), 17.47 MiB | 33.18 MiB/s Receiving objects: 10% (5472/54719), 17.47 MiB | 33.18 MiB/s Receiving objects: 11% (6020/54719), 17.47 MiB | 33.18 MiB/s Receiving objects: 12% (6567/54719), 17.47 MiB | 33.18 MiB/s Receiving objects: 13% (7114/54719), 17.47 MiB | 33.18 MiB/s Receiving objects: 13% (7545/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 14% (7661/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 15% (8208/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 16% (8756/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 17% (9303/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 18% (9850/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 19% (10397/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 20% (10944/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 21% (11491/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 22% (12039/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 23% (12586/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 24% (13133/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 25% (13680/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 26% (14227/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 27% (14775/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 28% (15322/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 29% (15869/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 30% (16416/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 31% (16963/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 32% (17511/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 33% (18058/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 34% (18605/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 35% (19152/54719), 34.56 MiB | 33.66 MiB/s Receiving objects: 36% (19699/54719), 51.71 MiB | 33.77 MiB/s Receiving objects: 37% (20247/54719), 51.71 MiB | 33.77 MiB/s Receiving objects: 38% (20794/54719), 51.71 MiB | 33.77 MiB/s Receiving objects: 39% (21341/54719), 51.71 MiB | 33.77 MiB/s Receiving objects: 39% (21872/54719), 69.08 MiB | 34.00 MiB/s Receiving objects: 40% (21888/54719), 69.08 MiB | 34.00 MiB/s Receiving objects: 41% (22435/54719), 69.08 MiB | 34.00 MiB/s Receiving objects: 42% (22982/54719), 69.08 MiB | 34.00 MiB/s Receiving objects: 43% (23530/54719), 69.08 MiB | 34.00 MiB/s Receiving objects: 44% (24077/54719), 87.01 MiB | 34.31 MiB/s Receiving objects: 45% (24624/54719), 87.01 MiB | 34.31 MiB/s Receiving objects: 46% (25171/54719), 87.01 MiB | 34.31 MiB/s Receiving objects: 47% (25718/54719), 87.01 MiB | 34.31 MiB/s Receiving objects: 48% (26266/54719), 87.01 MiB | 34.31 MiB/s Receiving objects: 48% (26400/54719), 104.09 MiB | 34.28 MiB/s Receiving objects: 49% (26813/54719), 121.48 MiB | 34.35 MiB/s Receiving objects: 49% (27101/54719), 121.48 MiB | 34.35 MiB/s Receiving objects: 50% (27360/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 51% (27907/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 52% (28454/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 53% (29002/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 54% (29549/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 55% (30096/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 56% (30643/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 57% (31190/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 58% (31738/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 59% (32285/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 60% (32832/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 61% (33379/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 62% (33926/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 63% (34473/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 64% (35021/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 65% (35568/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 66% (36115/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 67% (36662/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 68% (37209/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 69% (37757/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 70% (38304/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 71% (38851/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 72% (39398/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 73% (39945/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 74% (40493/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 75% (41040/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 76% (41587/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 77% (42134/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 78% (42681/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 79% (43229/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 80% (43776/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 81% (44323/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 82% (44870/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 83% (45417/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 84% (45964/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 85% (46512/54719), 138.79 MiB | 34.39 MiB/s Receiving objects: 86% (47059/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 87% (47606/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 88% (48153/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 89% (48700/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 90% (49248/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 91% (49795/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 92% (50342/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 93% (50889/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 94% (51436/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 95% (51984/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 96% (52531/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 97% (53078/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 98% (53625/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 99% (54172/54719), 156.78 MiB | 34.48 MiB/s remote: Total 54719 (delta 44), reused 28 (delta 28), pack-reused 54641 (from 3) Receiving objects: 100% (54719/54719), 156.78 MiB | 34.48 MiB/s Receiving objects: 100% (54719/54719), 168.30 MiB | 34.63 MiB/s, done. Resolving deltas: 0% (0/26547) Resolving deltas: 1% (266/26547) Resolving deltas: 2% (531/26547) Resolving deltas: 3% (797/26547) Resolving deltas: 4% (1063/26547) Resolving deltas: 5% (1328/26547) Resolving deltas: 6% (1593/26547) Resolving deltas: 7% (1860/26547) Resolving deltas: 8% (2125/26547) Resolving deltas: 9% (2390/26547) Resolving deltas: 10% (2655/26547) Resolving deltas: 11% (2921/26547) Resolving deltas: 12% (3186/26547) Resolving deltas: 13% (3452/26547) Resolving deltas: 14% (3717/26547) Resolving deltas: 15% (3983/26547) Resolving deltas: 16% (4248/26547) Resolving deltas: 17% (4513/26547) Resolving deltas: 18% (4779/26547) Resolving deltas: 19% (5044/26547) Resolving deltas: 20% (5311/26547) Resolving deltas: 21% (5575/26547) Resolving deltas: 22% (5841/26547) Resolving deltas: 23% (6106/26547) Resolving deltas: 24% (6373/26547) Resolving deltas: 25% (6637/26547) Resolving deltas: 26% (6903/26547) Resolving deltas: 27% (7168/26547) Resolving deltas: 28% (7434/26547) Resolving deltas: 29% (7699/26547) Resolving deltas: 30% (7965/26547) Resolving deltas: 31% (8230/26547) Resolving deltas: 32% (8496/26547) Resolving deltas: 33% (8761/26547) Resolving deltas: 34% (9026/26547) Resolving deltas: 35% (9292/26547) Resolving deltas: 36% (9557/26547) Resolving deltas: 37% (9825/26547) Resolving deltas: 38% (10090/26547) Resolving deltas: 39% (10354/26547) Resolving deltas: 40% (10622/26547) Resolving deltas: 41% (10885/26547) Resolving deltas: 42% (11150/26547) Resolving deltas: 43% (11416/26547) Resolving deltas: 44% (11681/26547) Resolving deltas: 45% (11947/26547) Resolving deltas: 46% (12212/26547) Resolving deltas: 47% (12478/26547) Resolving deltas: 48% (12744/26547) Resolving deltas: 49% (13009/26547) Resolving deltas: 50% (13274/26547) Resolving deltas: 51% (13539/26547) Resolving deltas: 52% (13805/26547) Resolving deltas: 53% (14070/26547) Resolving deltas: 54% (14337/26547) Resolving deltas: 55% (14601/26547) Resolving deltas: 56% (14868/26547) Resolving deltas: 57% (15132/26547) Resolving deltas: 58% (15398/26547) Resolving deltas: 59% (15665/26547) Resolving deltas: 60% (15929/26547) Resolving deltas: 61% (16194/26547) Resolving deltas: 62% (16460/26547) Resolving deltas: 63% (16725/26547) Resolving deltas: 64% (16991/26547) Resolving deltas: 65% (17256/26547) Resolving deltas: 66% (17522/26547) Resolving deltas: 67% (17787/26547) Resolving deltas: 68% (18052/26547) Resolving deltas: 69% (18318/26547) Resolving deltas: 70% (18583/26547) Resolving deltas: 71% (18849/26547) Resolving deltas: 72% (19114/26547) Resolving deltas: 73% (19380/26547) Resolving deltas: 74% (19645/26547) Resolving deltas: 75% (19911/26547) Resolving deltas: 76% (20176/26547) Resolving deltas: 77% (20442/26547) Resolving deltas: 78% (20707/26547) Resolving deltas: 79% (20973/26547) Resolving deltas: 80% (21238/26547) Resolving deltas: 81% (21505/26547) Resolving deltas: 82% (21769/26547) Resolving deltas: 83% (22036/26547) Resolving deltas: 84% (22300/26547) Resolving deltas: 85% (22565/26547) Resolving deltas: 86% (22831/26547) Resolving deltas: 87% (23096/26547) Resolving deltas: 88% (23362/26547) Resolving deltas: 89% (23627/26547) Resolving deltas: 90% (23893/26547) Resolving deltas: 91% (24158/26547) Resolving deltas: 92% (24424/26547) Resolving deltas: 93% (24690/26547) Resolving deltas: 94% (24955/26547) Resolving deltas: 95% (25221/26547) Resolving deltas: 96% (25486/26547) Resolving deltas: 97% (25751/26547) Resolving deltas: 98% (26019/26547) Resolving deltas: 99% (26282/26547) Resolving deltas: 100% (26547/26547) Resolving deltas: 100% (26547/26547), done. Fetching commit e923b9b20ed1f97069be27765c93a4cf20c28489 for refs/pull/1/merge ... $ /nix/store/bcnisk3ydfgv26v2gw3zlky24g00yww2-git-2.54.0/bin/git fetch origin e923b9b20ed1f97069be27765c93a4cf20c28489 remote: Enumerating objects: 1, done. remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 1 (from 1) Unpacking objects: 100% (1/1) 100% (1/1), 875 bytes | 875.00 KiB/s, done. From https://github.com/NorfairKing/dspy * branch e923b9b20ed1f97069be27765c93a4cf20c28489 -> FETCH_HEAD $ /nix/store/9ypz3flqsrl5xl495mm8h645gadjsxi1-coreutils-9.11/bin/timeout --kill-after=15s 7200s /nix/store/bcnisk3ydfgv26v2gw3zlky24g00yww2-git-2.54.0/bin/git -c advice.detachedHead=false checkout e923b9b20ed1f97069be27765c93a4cf20c28489 HEAD is now at e923b9b2 Merge 453ecf257a343742170b3e1cef20271508c3e6a4 into 498760149b230f402c56bece2aa45df6e1ba946b $ /nix/store/9ypz3flqsrl5xl495mm8h645gadjsxi1-coreutils-9.11/bin/timeout --kill-after=15s 7200s /nix/store/23zk8sgzamrbmj1fday15szf7f2279s8-nix-2.34.7/bin/nix --extra-experimental-features nix-command --extra-experimental-features flakes --log-format internal-json run .#packages.x86_64-linux.impure-test --print-build-logs  ============================= test session starts ============================== platform linux -- Python 3.13.13, pytest-8.4.2, pluggy-1.6.0 -- /nix/store/rn0r480kscbk8159qi3qjflil71kqglq-dspy-test-env/bin/python3.13 rootdir: /tmp/nix-ci-worker/stable-pwd/repo configfile: pyproject.toml plugins: anyio-4.13.0, asyncio-1.3.0, typeguard-4.5.1, mock-3.14.1 asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function collecting ...  collecting 6 items  collecting 288 items  collecting 604 items  collecting 1156 items  collected 1322 items / 1 skipped  tests/adapters/test_adapter_utils.py::test_parse_value_str_annotation PASSED [ 0%] tests/adapters/test_adapter_utils.py::test_parse_value_pydantic_types PASSED [ 0%] tests/adapters/test_adapter_utils.py::test_parse_value_basic_types PASSED [ 0%] tests/adapters/test_adapter_utils.py::test_parse_value_literal PASSED [ 0%] tests/adapters/test_adapter_utils.py::test_parse_value_union PASSED [ 0%] tests/adapters/test_adapter_utils.py::test_parse_value_json_repair PASSED [ 0%] tests/adapters/test_audio.py::test_normalize_audio_format[wav-wav] PASSED [ 0%] tests/adapters/test_audio.py::test_normalize_audio_format[mp3-mp3] PASSED [ 0%] tests/adapters/test_audio.py::test_normalize_audio_format[x-wav-wav] PASSED [ 0%] tests/adapters/test_audio.py::test_normalize_audio_format[x-mp3-mp3] PASSED [ 0%] tests/adapters/test_audio.py::test_normalize_audio_format[x-flac-flac] PASSED [ 0%] tests/adapters/test_audio.py::test_normalize_audio_format[my-x-format-my-x-format] PASSED [ 0%] tests/adapters/test_audio.py::test_normalize_audio_format[x-my-format-my-format] PASSED [ 0%] tests/adapters/test_audio.py::test_normalize_audio_format[-] PASSED [ 1%] tests/adapters/test_audio.py::test_normalize_audio_format[x--] PASSED [ 1%] tests/adapters/test_baml_adapter.py::test_baml_adapter_format_exact_messages_for_simple_signature_with_demo PASSED [ 1%] tests/adapters/test_baml_adapter.py::test_baml_adapter_format_exact_messages_with_nested_output PASSED [ 1%] tests/adapters/test_baml_adapter.py::test_baml_adapter_basic_schema_generation PASSED [ 1%] tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_optional_fields PASSED [ 1%] tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_primitive_types PASSED [ 1%] tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_lists_with_bracket_notation PASSED [ 1%] tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_complex_nested_models PASSED [ 1%] tests/adapters/test_baml_adapter.py::test_baml_adapter_raise_error_on_circular_references PASSED [ 1%] tests/adapters/test_baml_adapter.py::test_baml_adapter_formats_pydantic_inputs_as_clean_json PASSED [ 1%] tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_mixed_input_types PASSED [ 1%] tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_schema_generation_errors_gracefully PASSED [ 1%] tests/adapters/test_baml_adapter.py::test_baml_adapter_raises_on_missing_fields PASSED [ 2%] tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_type_casting_errors PASSED [ 2%] tests/adapters/test_baml_adapter.py::test_baml_adapter_with_images PASSED [ 2%] tests/adapters/test_baml_adapter.py::test_baml_adapter_with_tools PASSED [ 2%] tests/adapters/test_baml_adapter.py::test_baml_adapter_with_code PASSED [ 2%] tests/adapters/test_baml_adapter.py::test_baml_adapter_with_conversation_history PASSED [ 2%] tests/adapters/test_baml_adapter.py::test_baml_vs_json_adapter_token_efficiency PASSED [ 2%] tests/adapters/test_baml_adapter.py::test_baml_vs_json_adapter_functional_compatibility PASSED [ 2%] tests/adapters/test_baml_adapter.py::test_baml_adapter_async_functionality PASSED [ 2%] tests/adapters/test_baml_adapter.py::test_baml_adapter_with_field_aliases PASSED [ 2%] tests/adapters/test_baml_adapter.py::test_baml_adapter_field_alias_without_description PASSED [ 2%] tests/adapters/test_baml_adapter.py::test_baml_adapter_multiple_pydantic_input_fields PASSED [ 2%] tests/adapters/test_base_type.py::test_basic_extract_custom_type_from_annotation PASSED [ 2%] tests/adapters/test_base_type.py::test_extract_custom_type_from_annotation_with_nested_type PASSED [ 3%] tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-two-Literal['one', 'two', 'three"']-Literal['four', 'five', 'six"']] PASSED [ 3%] tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-she's here-Literal["she's here", 'okay', 'test']-Literal['done', "maybe'soon", 'later']] PASSED [ 3%] tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-another-Literal['both"and\\'', 'another']-Literal['yet"another\\'', 'plain']] PASSED [ 3%] tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-foo-Literal['foo', 'bar']-Literal['baz', 'qux']] PASSED [ 3%] tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-bar-Literal[1, 'bar']-Literal[True, 3, 'foo']] PASSED [ 3%] tests/adapters/test_chat_adapter.py::test_chat_adapter_sync_call PASSED [ 3%] tests/adapters/test_chat_adapter.py::test_chat_adapter_async_call PASSED [ 3%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_for_simple_signature PASSED [ 3%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_demo_and_typed_outputs PASSED [ 3%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_nested_pydantic_models PASSED [ 3%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_incomplete_demo PASSED [ 3%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_history PASSED [ 3%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_list_value_for_string_input PASSED [ 4%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_literal_output PASSED [ 4%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_multimodal_custom_type_inputs PASSED [ 4%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_history_demo_pydantic_tools_and_image PASSED [ 4%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_base_custom_type_input PASSED [ 4%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_citations_output_demo PASSED [ 4%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_and_lm_kwargs_with_native_citations PASSED [ 4%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_preserves_passthrough_lm_kwargs PASSED [ 4%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_and_lm_kwargs_with_native_reasoning PASSED [ 4%] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_calling_still_enables_native_reasoning PASSED [ 4%] tests/adapters/test_chat_adapter.py::test_chat_adapter_nonnative_strips_native_tool_kwargs PASSED [ 4%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_reasoning_and_code_outputs PASSED [ 4%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_and_lm_kwargs_with_native_tool_calling PASSED [ 4%] tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_can_request_parallel_tool_calls[adapter0] PASSED [ 4%] tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_can_request_parallel_tool_calls[adapter1] PASSED [ 5%] tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_can_request_parallel_tool_calls[adapter2] PASSED [ 5%] tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_respects_lm_kwargs_parallel_tool_call_override PASSED [ 5%] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_replay PASSED [ 5%] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_replays_parallel_tool_results PASSED [ 5%] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_empty_user_message PASSED [ 5%] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_unmatched_tool_calls[call_1-None] PASSED [ 5%] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_unmatched_tool_calls[call_1-tool_call_results1] PASSED [ 5%] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_unmatched_tool_calls[None-tool_call_results2] PASSED [ 5%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_non_native_tool_history PASSED [ 5%] tests/adapters/test_chat_adapter.py::test_non_native_tool_history_remains_text_based[adapter0] PASSED [ 5%] tests/adapters/test_chat_adapter.py::test_non_native_tool_history_remains_text_based[adapter1] PASSED [ 5%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_accepts_custom_history_formatter_returning_messages_only PASSED [ 5%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_tool_input PASSED [ 6%] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_kitchen_sink PASSED [ 6%] tests/adapters/test_chat_adapter.py::test_chat_adapter_with_pydantic_models PASSED [ 6%] tests/adapters/test_chat_adapter.py::test_chat_adapter_signature_information PASSED [ 6%] tests/adapters/test_chat_adapter.py::test_chat_adapter_exception_raised_on_failure PASSED [ 6%] tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image PASSED [ 6%] tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image_with_few_shot_examples PASSED [ 6%] tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image_with_nested_images PASSED [ 6%] tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image_with_few_shot_examples_with_nested_images PASSED [ 6%] tests/adapters/test_chat_adapter.py::test_chat_adapter_with_tool PASSED [ 6%] tests/adapters/test_chat_adapter.py::test_chat_adapter_with_code PASSED [ 6%] tests/adapters/test_chat_adapter.py::test_code_output_field_omits_json_schema_in_prompt PASSED [ 6%] tests/adapters/test_chat_adapter.py::test_citations_output_field_keeps_json_schema_in_prompt PASSED [ 6%] tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_conversation_history PASSED [ 7%] tests/adapters/test_chat_adapter.py::test_chat_adapter_fallback_to_json_adapter_on_exception PASSED [ 7%] tests/adapters/test_chat_adapter.py::test_chat_adapter_fallback_preserves_native_function_calling_flag PASSED [ 7%] tests/adapters/test_chat_adapter.py::test_chat_adapter_respects_use_json_adapter_fallback_flag PASSED [ 7%] tests/adapters/test_chat_adapter.py::test_chat_adapter_fallback_to_json_adapter_on_exception_async PASSED [ 7%] tests/adapters/test_chat_adapter.py::test_chat_adapter_async_fallback_preserves_native_function_calling_flag PASSED [ 7%] tests/adapters/test_chat_adapter.py::test_chat_adapter_toolcalls_native_function_calling PASSED [ 7%] tests/adapters/test_chat_adapter.py::test_chat_adapter_toolcalls_vague_match PASSED [ 7%] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_reasoning PASSED [ 7%] tests/adapters/test_chat_adapter.py::test_chat_adapter_parses_float_with_underscores PASSED [ 7%] tests/adapters/test_chat_adapter.py::test_format_system_message PASSED [ 7%] tests/adapters/test_chat_adapter.py::test_null_content_raises_adapter_parse_error PASSED [ 7%] tests/adapters/test_chat_adapter.py::test_empty_string_content_raises_adapter_parse_error PASSED [ 7%] tests/adapters/test_chat_adapter.py::test_tool_call_with_null_content_does_not_raise PASSED [ 8%] tests/adapters/test_chat_adapter.py::test_tool_call_with_unstructured_content_does_not_raise PASSED [ 8%] tests/adapters/test_chat_adapter.py::test_tool_call_with_structured_content_preserves_other_outputs PASSED [ 8%] tests/adapters/test_chat_adapter.py::test_provider_tool_calls_preserve_id_and_repair_arguments PASSED [ 8%] tests/adapters/test_citation.py::test_citation_validate_input PASSED [ 8%] tests/adapters/test_citation.py::test_citations_in_nested_type PASSED [ 8%] tests/adapters/test_citation.py::test_citation_with_all_fields PASSED [ 8%] tests/adapters/test_citation.py::test_citation_format PASSED [ 8%] tests/adapters/test_citation.py::test_citations_format PASSED [ 8%] tests/adapters/test_citation.py::test_citations_from_dict_list PASSED [ 8%] tests/adapters/test_citation.py::test_citations_postprocessing PASSED [ 8%] tests/adapters/test_citation.py::test_citation_extraction_from_lm_response PASSED [ 8%] tests/adapters/test_code.py::test_code_validate_input PASSED [ 8%] tests/adapters/test_code.py::test_code_in_nested_type PASSED [ 9%] tests/adapters/test_code.py::test_code_with_language PASSED [ 9%] tests/adapters/test_code.py::test_code_parses_from_dirty_code PASSED [ 9%] tests/adapters/test_document.py::test_document_validate_input PASSED [ 9%] tests/adapters/test_document.py::test_document_in_nested_type PASSED [ 9%] tests/adapters/test_document.py::test_document_with_all_fields PASSED [ 9%] tests/adapters/test_document.py::test_document_format PASSED [ 9%] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_for_simple_signature PASSED [ 9%] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_demo_and_typed_output PASSED [ 9%] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_described_and_bool_outputs PASSED [ 9%] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_history_demo_pydantic_tools_and_image PASSED [ 9%] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_int_and_mapping_outputs PASSED [ 9%] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_literal_and_enum_outputs PASSED [ 9%] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_nested_pydantic_output PASSED [ 9%] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_incomplete_demo PASSED [ 10%] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_and_lm_kwargs_with_native_tool_calling PASSED [ 10%] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_tool_calls_output_demo PASSED [ 10%] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_non_native_tool_result_history_field PASSED [ 10%] tests/adapters/test_json_adapter.py::test_json_adapter_passes_structured_output_when_supported_by_model PASSED [ 10%] tests/adapters/test_json_adapter.py::test_json_adapter_not_using_structured_outputs_when_not_supported_by_model PASSED [ 10%] tests/adapters/test_json_adapter.py::test_json_adapter_with_structured_outputs_does_not_mutate_original_signature PASSED [ 10%] tests/adapters/test_json_adapter.py::test_json_adapter_sync_call PASSED [ 10%] tests/adapters/test_json_adapter.py::test_json_adapter_async_call PASSED [ 10%] tests/adapters/test_json_adapter.py::test_json_adapter_on_pydantic_model PASSED [ 10%] tests/adapters/test_json_adapter.py::test_json_adapter_parse_raise_error_on_mismatch_fields PASSED [ 10%] tests/adapters/test_json_adapter.py::test_json_adapter_formats_image PASSED [ 10%] tests/adapters/test_json_adapter.py::test_json_adapter_formats_image_with_few_shot_examples PASSED [ 10%] tests/adapters/test_json_adapter.py::test_json_adapter_formats_image_with_nested_images PASSED [ 11%] tests/adapters/test_json_adapter.py::test_json_adapter_formats_with_nested_documents PASSED [ 11%] tests/adapters/test_json_adapter.py::test_json_adapter_formats_image_with_few_shot_examples_with_nested_images PASSED [ 11%] tests/adapters/test_json_adapter.py::test_json_adapter_with_tool PASSED [ 11%] tests/adapters/test_json_adapter.py::test_json_adapter_with_code PASSED [ 11%] tests/adapters/test_json_adapter.py::test_json_adapter_formats_conversation_history PASSED [ 11%] tests/adapters/test_json_adapter.py::test_json_adapter_on_pydantic_model_async PASSED [ 11%] tests/adapters/test_json_adapter.py::test_json_adapter_does_not_fallback_to_json_mode_on_structured_output_lm_error PASSED [ 11%] tests/adapters/test_json_adapter.py::test_json_adapter_json_mode_no_structured_outputs PASSED [ 11%] tests/adapters/test_json_adapter.py::test_json_adapter_json_mode_no_structured_outputs_async PASSED [ 11%] tests/adapters/test_json_adapter.py::test_json_adapter_does_not_fallback_to_json_mode_on_structured_output_lm_error_async PASSED [ 11%] tests/adapters/test_json_adapter.py::test_error_message_on_json_adapter_failure PASSED [ 11%] tests/adapters/test_json_adapter.py::test_error_message_on_json_adapter_failure_async PASSED [ 11%] tests/adapters/test_json_adapter.py::test_json_adapter_toolcalls_native_function_calling PASSED [ 12%] tests/adapters/test_json_adapter.py::test_json_adapter_toolcalls_no_native_function_calling PASSED [ 12%] tests/adapters/test_json_adapter.py::test_json_adapter_native_reasoning PASSED [ 12%] tests/adapters/test_json_adapter.py::test_json_adapter_with_responses_api PASSED [ 12%] tests/adapters/test_json_adapter.py::test_format_system_message PASSED [ 12%] tests/adapters/test_reasoning.py::test_reasoning_basic_operations PASSED [ 12%] tests/adapters/test_reasoning.py::test_reasoning_concatenation PASSED [ 12%] tests/adapters/test_reasoning.py::test_reasoning_string_methods PASSED [ 12%] tests/adapters/test_reasoning.py::test_reasoning_with_chain_of_thought PASSED [ 12%] tests/adapters/test_reasoning.py::test_reasoning_error_message PASSED [ 12%] tests/adapters/test_tool.py::test_basic_initialization PASSED [ 12%] tests/adapters/test_tool.py::test_tool_from_function PASSED [ 12%] tests/adapters/test_tool.py::test_tool_from_class PASSED [ 12%] tests/adapters/test_tool.py::test_tool_from_function_with_pydantic PASSED [ 13%] tests/adapters/test_tool.py::test_tool_from_function_with_pydantic_nesting PASSED [ 13%] tests/adapters/test_tool.py::test_tool_callable PASSED [ 13%] tests/adapters/test_tool.py::test_tool_with_pydantic_callable PASSED [ 13%] tests/adapters/test_tool.py::test_invalid_function_call PASSED [ 13%] tests/adapters/test_tool.py::test_parameter_desc PASSED [ 13%] tests/adapters/test_tool.py::test_tool_with_default_args_without_type_hints PASSED [ 13%] tests/adapters/test_tool.py::test_tool_call_parses_args PASSED [ 13%] tests/adapters/test_tool.py::test_tool_call_parses_nested_list_of_pydantic_model PASSED [ 13%] tests/adapters/test_tool.py::test_tool_call_kwarg PASSED [ 13%] tests/adapters/test_tool.py::test_tool_str PASSED [ 13%] tests/adapters/test_tool.py::test_async_tool_from_function PASSED [ 13%] tests/adapters/test_tool.py::test_async_tool_with_pydantic PASSED [ 13%] tests/adapters/test_tool.py::test_async_tool_with_complex_pydantic PASSED [ 13%] tests/adapters/test_tool.py::test_async_tool_invalid_call PASSED [ 14%] tests/adapters/test_tool.py::test_async_tool_with_kwargs PASSED [ 14%] tests/adapters/test_tool.py::test_async_concurrent_calls PASSED [ 14%] tests/adapters/test_tool.py::test_async_tool_call_in_sync_mode PASSED [ 14%] tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data0-expected0] PASSED [ 14%] tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data1-expected1] PASSED [ 14%] tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data2-expected2] PASSED [ 14%] tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data3-expected3] PASSED [ 14%] tests/adapters/test_tool.py::test_tool_calls_format_from_dict_list PASSED [ 14%] tests/adapters/test_tool.py::test_tool_calls_preserves_ids_from_dict_list_and_format_omits_ids PASSED [ 14%] tests/adapters/test_tool.py::test_tool_calls_json_schema_omits_internal_id_field PASSED [ 14%] tests/adapters/test_tool.py::test_tool_calls_can_carry_results_without_formatting_them_for_lm PASSED [ 14%] tests/adapters/test_tool.py::test_tool_call_results_from_tool_calls_and_values PASSED [ 14%] tests/adapters/test_tool.py::test_tool_call_results_history_serialization_round_trip PASSED [ 15%] tests/adapters/test_tool.py::test_tool_call_results_can_round_trip_as_native_tool_result_message PASSED [ 15%] tests/adapters/test_tool.py::test_toolcalls_vague_match PASSED [ 15%] tests/adapters/test_tool.py::test_tool_convert_input_schema_to_tool_args_no_input_params PASSED [ 15%] tests/adapters/test_tool.py::test_tool_convert_input_schema_to_tool_args_lang_chain PASSED [ 15%] tests/adapters/test_tool.py::test_tool_call_execute PASSED [ 15%] tests/adapters/test_tool.py::test_tool_call_execute_with_local_functions PASSED [ 15%] tests/adapters/test_two_step_adapter.py::test_two_step_adapter_format_exact_messages_for_simple_signature_with_demo PASSED [ 15%] tests/adapters/test_two_step_adapter.py::test_two_step_adapter_format_exact_messages_with_typed_outputs PASSED [ 15%] tests/adapters/test_two_step_adapter.py::test_two_step_adapter_call PASSED [ 15%] tests/adapters/test_two_step_adapter.py::test_two_step_adapter_async_call PASSED [ 15%] tests/adapters/test_two_step_adapter.py::test_two_step_adapter_parse PASSED [ 15%] tests/adapters/test_two_step_adapter.py::test_two_step_adapter_parse_errors PASSED [ 15%] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_and_parse_basic PASSED [ 16%] tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_multiple_fields PASSED [ 16%] tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_raises_on_missing_field PASSED [ 16%] tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_casts_types PASSED [ 16%] tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_raises_on_type_error PASSED [ 16%] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_and_parse_nested_model PASSED [ 16%] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_and_parse_list_of_models PASSED [ 16%] tests/adapters/test_xml_adapter.py::test_xml_adapter_with_tool_like_output PASSED [ 16%] tests/adapters/test_xml_adapter.py::test_xml_adapter_formats_nested_images PASSED [ 16%] tests/adapters/test_xml_adapter.py::test_xml_adapter_with_code PASSED [ 16%] tests/adapters/test_xml_adapter.py::test_xml_adapter_full_prompt PASSED [ 16%] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_for_simple_signature PASSED [ 16%] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_non_native_tool_result_history_field PASSED [ 16%] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_for_two_input_signature PASSED [ 17%] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_demo_and_typed_output PASSED [ 17%] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_history_demo_pydantic_tools_and_image PASSED [ 17%] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_nested_pydantic_output PASSED [ 17%] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_incomplete_demo PASSED [ 17%] tests/adapters/test_xml_adapter.py::test_format_system_message PASSED [ 17%] tests/callback/test_callback.py::test_callback_injection[args0-kwargs0] PASSED [ 17%] tests/callback/test_callback.py::test_callback_injection[args1-kwargs1] PASSED [ 17%] tests/callback/test_callback.py::test_callback_injection[args2-kwargs2] PASSED [ 17%] tests/callback/test_callback.py::test_callback_injection[args3-kwargs3] PASSED [ 17%] tests/callback/test_callback.py::test_callback_injection_local PASSED [ 17%] tests/callback/test_callback.py::test_callback_error_handling PASSED [ 17%] tests/callback/test_callback.py::test_multiple_callbacks PASSED [ 17%] tests/callback/test_callback.py::test_callback_complex_module PASSED [ 18%] tests/callback/test_callback.py::test_callback_async_module PASSED [ 18%] tests/callback/test_callback.py::test_tool_calls PASSED [ 18%] tests/callback/test_callback.py::test_active_id PASSED [ 18%] tests/clients/test_cache.py::test_initialization PASSED [ 18%] tests/clients/test_cache.py::test_invalid_cache_initialization PASSED [ 18%] tests/clients/test_cache.py::test_cache_key_generation PASSED [ 18%] tests/clients/test_cache.py::test_put_and_get PASSED [ 18%] tests/clients/test_cache.py::test_cache_miss PASSED [ 18%] tests/clients/test_cache.py::test_unserializable_key PASSED [ 18%] tests/clients/test_cache.py::test_reset_memory_cache PASSED [ 18%] tests/clients/test_cache.py::test_save_and_load_memory_cache PASSED [ 18%] tests/clients/test_cache.py::test_request_cache_decorator PASSED [ 18%] tests/clients/test_cache.py::test_request_cache_decorator_with_ignored_args_for_cache_key PASSED [ 18%] tests/clients/test_cache.py::test_request_cache_decorator_async PASSED [ 19%] tests/clients/test_cache.py::test_cache_consistency_with_lm_call_modifies_the_request PASSED [ 19%] tests/clients/test_cache.py::test_cache_fallback_on_restricted_environment PASSED [ 19%] tests/clients/test_cache.py::test_cache_init_with_disk_disabled_and_none_dir PASSED [ 19%] tests/clients/test_cache.py::test_model_response_roundtrip_in_restricted_mode PASSED [ 19%] tests/clients/test_cache.py::test_registered_dataclass_roundtrip_in_restricted_mode PASSED [ 19%] tests/clients/test_cache.py::test_configure_cache_registers_safe_types PASSED [ 19%] tests/clients/test_cache.py::test_corrupt_disk_entries_return_none PASSED [ 19%] tests/clients/test_cache.py::test_restricted_and_unrestricted_share_wire_format PASSED [ 19%] tests/clients/test_cache.py::test_unlisted_type_rejected_on_read PASSED [ 19%] tests/clients/test_cache.py::test_safe_types_rejects_non_types PASSED [ 19%] tests/clients/test_databricks.py::test_create_directory_in_databricks_unity_catalog SKIPPED [ 19%] tests/clients/test_databricks.py::test_create_finetuning_job SKIPPED [ 19%] tests/clients/test_databricks.py::test_deploy_finetuned_model SKIPPED [ 20%] tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[dict] PASSED [ 20%] tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[list] PASSED [ 20%] tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[str] PASSED [ 20%] tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[int] PASSED [ 20%] tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[tuple] PASSED [ 20%] tests/clients/test_disk_serialization.py::test_pydantic_roundtrip PASSED [ 20%] tests/clients/test_disk_serialization.py::test_dataclass_roundtrip PASSED [ 20%] tests/clients/test_disk_serialization.py::test_ndarray_roundtrip PASSED [ 20%] tests/clients/test_disk_serialization.py::test_litellm_response_roundtrip PASSED [ 20%] tests/clients/test_disk_serialization.py::test_text_completion_response_roundtrip PASSED [ 20%] tests/clients/test_disk_serialization.py::test_embedding_response_roundtrip PASSED [ 20%] tests/clients/test_disk_serialization.py::test_responses_api_roundtrip PASSED [ 20%] tests/clients/test_disk_serialization.py::test_tool_call_response_roundtrip PASSED [ 21%] tests/clients/test_disk_serialization.py::test_cached_tool_call_normalizes_after_fresh_process_roundtrip PASSED [ 21%] tests/clients/test_disk_serialization.py::test_cached_responses_function_call_normalizes_after_fresh_process_roundtrip PASSED [ 21%] tests/clients/test_disk_serialization.py::test_unlisted_type_blocked_on_read PASSED [ 21%] tests/clients/test_disk_serialization.py::test_allowlists_are_isolated PASSED [ 21%] tests/clients/test_disk_serialization.py::test_numpy_ctypeslib_blocked PASSED [ 21%] tests/clients/test_disk_serialization.py::test_restricted_load_rejects_unknown_type PASSED [ 21%] tests/clients/test_disk_serialization.py::test_corrupt_pickle_raises_deserialization_error PASSED [ 21%] tests/clients/test_disk_serialization.py::test_nested_class_safe_types PASSED [ 21%] tests/clients/test_disk_serialization.py::test_restricted_load_allows_builtin_types PASSED [ 21%] tests/clients/test_disk_serialization.py::test_all_cached_lm_types_roundtrip PASSED [ 21%] tests/clients/test_embedding.py::test_litellm_embedding PASSED [ 21%] tests/clients/test_embedding.py::test_callable_embedding PASSED [ 21%] tests/clients/test_embedding.py::test_callable_numpy_embedding_persists_to_disk PASSED [ 22%] tests/clients/test_embedding.py::test_invalid_model_type PASSED [ 22%] tests/clients/test_embedding.py::test_async_embedding PASSED [ 22%] tests/clients/test_embedding.py::test_call_caching_false_overrides_instance_true PASSED [ 22%] tests/clients/test_embedding.py::test_call_caching_true_overrides_instance_false PASSED [ 22%] tests/clients/test_embedding.py::test_acall_caching_false_overrides_instance_true PASSED [ 22%] tests/clients/test_embedding.py::test_acall_caching_true_overrides_instance_false PASSED [ 22%] tests/clients/test_inspect_global_history.py::test_inspect_history_basic PASSED [ 22%] tests/clients/test_inspect_global_history.py::test_inspect_history_renders_message_tool_calls PASSED [ 22%] tests/clients/test_inspect_global_history.py::test_inspect_history_renders_output_tool_calls_without_text PASSED [ 22%] tests/clients/test_inspect_global_history.py::test_inspect_history_with_n PASSED [ 22%] tests/clients/test_inspect_global_history.py::test_inspect_empty_history PASSED [ 22%] tests/clients/test_inspect_global_history.py::test_inspect_history_n_larger_than_history PASSED [ 22%] tests/clients/test_lazy_litellm_import.py::test_import_dspy_does_not_import_litellm PASSED [ 22%] tests/clients/test_lazy_litellm_import.py::test_lm_litellm_use_raises_helpful_error_without_litellm PASSED [ 23%] tests/clients/test_lazy_litellm_import.py::test_embedder_litellm_use_raises_helpful_error_without_litellm PASSED [ 23%] tests/clients/test_lazy_litellm_import.py::test_concurrent_lm_first_use_materializes_litellm_once PASSED [ 23%] tests/clients/test_lm.py::test_chat_lms_can_be_queried PASSED [ 23%] tests/clients/test_lm.py::test_dspy_cache PASSED [ 23%] tests/clients/test_lm.py::test_disabled_cache_skips_cache_key PASSED [ 23%] tests/clients/test_lm.py::test_rollout_id_bypasses_cache PASSED [ 23%] tests/clients/test_lm.py::test_zero_temperature_rollout_warns_once PASSED [ 23%] tests/clients/test_lm.py::test_rollout_id_with_default_temperature_does_not_warn PASSED [ 23%] tests/clients/test_lm.py::test_text_lms_can_be_queried PASSED [ 23%] tests/clients/test_lm.py::test_lm_calls_support_callables PASSED [ 23%] tests/clients/test_lm.py::test_lm_calls_support_pydantic_models PASSED [ 23%] tests/clients/test_lm.py::test_lm_wraps_litellm_errors_with_metadata PASSED [ 23%] tests/clients/test_lm.py::test_lm_wraps_litellm_context_window_error PASSED [ 24%] tests/clients/test_lm.py::test_lm_wraps_unknown_boundary_error_as_unexpected_error PASSED [ 24%] tests/clients/test_lm.py::test_lm_preserves_existing_lm_error_without_self_cause PASSED [ 24%] tests/clients/test_lm.py::test_lm_preserves_existing_lm_error_without_self_cause_async PASSED [ 24%] tests/clients/test_lm.py::test_retry_number_set_correctly PASSED [ 24%] tests/clients/test_lm.py::test_retry_made_on_system_errors PASSED [ 24%] tests/clients/test_lm.py::test_reasoning_model_token_parameter PASSED [ 24%] tests/clients/test_lm.py::test_reasoning_model_requirements[openai/o1] PASSED [ 24%] tests/clients/test_lm.py::test_reasoning_model_requirements[openai/gpt-5-nano] PASSED [ 24%] tests/clients/test_lm.py::test_reasoning_model_requirements[openai/gpt-5-mini] PASSED [ 24%] tests/clients/test_lm.py::test_gpt_5_chat_not_reasoning_model PASSED [ 24%] tests/clients/test_lm.py::test_base_lm_init_uses_lm_defaults_and_isolates_callback_list PASSED [ 24%] tests/clients/test_lm.py::test_base_lm_forward_contract_defaults_to_legacy PASSED [ 24%] tests/clients/test_lm.py::test_base_lm_forward_contract_accepts_explicit_values PASSED [ 25%] tests/clients/test_lm.py::test_base_lm_forward_contract_rejects_unknown_values PASSED [ 25%] tests/clients/test_lm.py::test_base_lm_validates_typed_lm_response PASSED [ 25%] tests/clients/test_lm.py::test_base_lm_warns_when_inherited_legacy_forward_returns_lm_response PASSED [ 25%] tests/clients/test_lm.py::test_base_lm_errors_when_explicit_legacy_forward_returns_lm_response PASSED [ 25%] tests/clients/test_lm.py::test_base_lm_inherited_legacy_forward_returning_lm_response_errors_on_direct_call PASSED [ 25%] tests/clients/test_lm.py::test_base_lm_default_call_keeps_legacy_outputs PASSED [ 25%] tests/clients/test_lm.py::test_base_lm_experimental_call_returns_lm_response_through_legacy_bridge PASSED [ 25%] tests/clients/test_lm.py::test_base_lm_explicit_lm_request_returns_lm_response_without_experimental PASSED [ 25%] tests/clients/test_lm.py::test_base_lm_legacy_bridge_records_typed_history_and_usage_once PASSED [ 25%] tests/clients/test_lm.py::test_base_lm_typed_forward_contract_uses_lm_request PASSED [ 25%] tests/clients/test_lm.py::test_base_lm_typed_forward_contract_rejects_non_lm_response_at_call_time PASSED [ 25%] tests/clients/test_lm.py::test_base_lm_request_call_rejects_mixed_inputs PASSED [ 25%] tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_system_user_and_assistant_turns[current_lm] PASSED [ 26%] tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_system_user_and_assistant_turns[typed_lm] PASSED [ 26%] tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_tool_call_transcripts[current_lm] PASSED [ 26%] tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_tool_call_transcripts[typed_lm] PASSED [ 26%] tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_can_reuse_lm_response_as_assistant_turn[current_lm] PASSED [ 26%] tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_can_reuse_lm_response_as_assistant_turn[typed_lm] PASSED [ 26%] tests/clients/test_lm.py::test_base_lm_async_explicit_lm_request_returns_lm_response PASSED [ 26%] tests/clients/test_lm.py::test_base_lm_tracks_usage_for_custom_subclasses PASSED [ 26%] tests/clients/test_lm.py::test_base_lm_copy_is_shallow_runtime_copy_with_isolated_dspy_state PASSED [ 26%] tests/clients/test_lm.py::test_dump_state PASSED [ 26%] tests/clients/test_lm.py::test_reasoning_model_dump_state_uses_constructor_max_tokens PASSED [ 26%] tests/clients/test_lm.py::test_dump_state_preserves_enabled_developer_role PASSED [ 26%] tests/clients/test_lm.py::test_dump_state_ignores_internal_class_marker_kwarg PASSED [ 26%] tests/clients/test_lm.py::test_load_state PASSED [ 27%] tests/clients/test_lm.py::test_reasoning_model_load_state_round_trips_canonical_state PASSED [ 27%] tests/clients/test_lm.py::test_reasoning_model_load_state_accepts_max_completion_tokens_alias PASSED [ 27%] tests/clients/test_lm.py::test_lm_load_state_forwards_allow_custom_lm_class PASSED [ 27%] tests/clients/test_lm.py::test_exponential_backoff_retry PASSED [ 27%] tests/clients/test_lm.py::test_logprobs_included_when_requested PASSED [ 27%] tests/clients/test_lm.py::test_async_lm_call PASSED [ 27%] tests/clients/test_lm.py::test_async_lm_call_with_cache PASSED [ 27%] tests/clients/test_lm.py::test_lm_history_size_limit PASSED [ 27%] tests/clients/test_lm.py::test_disable_history PASSED [ 27%] tests/clients/test_lm.py::test_responses_api PASSED [ 27%] tests/clients/test_lm.py::test_lm_replaces_system_with_developer_role PASSED [ 27%] tests/clients/test_lm.py::test_responses_api_tool_calls PASSED [ 27%] tests/clients/test_lm.py::test_reasoning_effort_responses_api PASSED [ 27%] tests/clients/test_lm.py::test_call_reasoning_model_with_chat_api PASSED [ 28%] tests/clients/test_lm.py::test_api_key_not_saved_in_json PASSED [ 28%] tests/clients/test_lm.py::test_responses_api_converts_images_correctly PASSED [ 28%] tests/clients/test_lm.py::test_responses_api_converts_files_correctly PASSED [ 28%] tests/clients/test_lm.py::test_responses_api_preserves_multi_message_structure PASSED [ 28%] tests/clients/test_lm.py::test_responses_api_with_image_input PASSED [ 28%] tests/clients/test_lm.py::test_responses_api_with_pydantic_model_input PASSED [ 28%] tests/clients/test_lm.py::test_responses_api_with_none_usage PASSED [ 28%] tests/clients/test_lm.py::test_responses_api_with_none_usage_async PASSED [ 28%] tests/clients/test_lm.py::test_streaming_passes_headers_correctly PASSED [ 28%] tests/clients/test_lm_direct_live.py::test_live_openai_chat_direct_system_user_assistant_multiturn SKIPPED [ 28%] tests/clients/test_lm_direct_live.py::test_live_openai_chat_direct_tool_call_transcript SKIPPED [ 28%] tests/clients/test_lm_direct_live.py::test_live_openai_chat_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 28%] tests/clients/test_lm_direct_live.py::test_live_openai_responses_direct_system_user_assistant_multiturn SKIPPED [ 29%] tests/clients/test_lm_direct_live.py::test_live_openai_responses_direct_tool_call_transcript SKIPPED [ 29%] tests/clients/test_lm_direct_live.py::test_live_openai_responses_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 29%] tests/clients/test_lm_direct_live.py::test_live_anthropic_chat_direct_system_user_assistant_multiturn SKIPPED [ 29%] tests/clients/test_lm_direct_live.py::test_live_anthropic_chat_direct_tool_call_transcript SKIPPED [ 29%] tests/clients/test_lm_direct_live.py::test_live_anthropic_chat_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 29%] tests/clients/test_lm_direct_live.py::test_live_gemini_chat_direct_system_user_assistant_multiturn SKIPPED [ 29%] tests/clients/test_lm_direct_live.py::test_live_gemini_chat_direct_tool_call_transcript SKIPPED [ 29%] tests/clients/test_lm_direct_live.py::test_live_gemini_chat_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 29%] tests/clients/test_lm_local.py::test_command_with_spaces_in_path PASSED [ 29%] tests/clients/test_lm_local.py::test_command_construction_prevents_injection PASSED [ 29%] tests/clients/test_lm_local.py::test_command_is_list_not_string PASSED [ 29%] tests/core/test_types.py::test_message_content_and_tool_calls_normalize_for_dspy_history_surface PASSED [ 29%] tests/core/test_types.py::test_tool_result_content_none_normalizes_to_empty_parts PASSED [ 30%] tests/core/test_types.py::test_audio_content_accepts_url_and_history_preserves_url PASSED [ 30%] tests/core/test_types.py::test_audio_content_defaults_null_format_to_wav PASSED [ 30%] tests/core/test_types.py::test_image_content_requires_mapping_with_url PASSED [ 30%] tests/core/test_types.py::test_video_data_round_trips_through_history_messages PASSED [ 30%] tests/core/test_types.py::test_document_source_url_stays_url_and_round_trips_through_history_messages PASSED [ 30%] tests/core/test_types.py::test_config_extensions_flatten_when_converted_to_legacy_kwargs PASSED [ 30%] tests/core/test_types.py::test_lm_kwargs_aliases_normalize_for_existing_dspy_lm_callers PASSED [ 30%] tests/core/test_types.py::test_nested_config_aliases_remain_supported_for_existing_interfaces PASSED [ 30%] tests/core/test_types.py::test_usage_normalizes_existing_user_visible_token_aliases PASSED [ 30%] tests/core/test_types.py::test_default_config_does_not_serialize_empty_stop_sequences PASSED [ 30%] tests/core/test_types.py::test_history_entry_exposes_existing_dict_style_properties PASSED [ 30%] tests/core/test_types.py::test_response_rejects_empty_outputs PASSED [ 30%] tests/core/test_types.py::test_output_to_value_preserves_redacted_thinking_part PASSED [ 31%] tests/core/test_types.py::test_stream_event_indices_must_be_non_negative PASSED [ 31%] tests/core/test_types.py::test_stream_builder_rejects_sparse_output_indices PASSED [ 31%] tests/core/test_types.py::test_stream_builder_rejects_sparse_part_indices PASSED [ 31%] tests/core/test_types.py::test_stream_builder_rejects_delta_type_changes PASSED [ 31%] tests/core/test_types.py::test_stream_builder_rejects_incomplete_tool_call_arguments PASSED [ 31%] tests/datasets/test_dataset.py::test_input_keys SKIPPED (need --extra option to run) [ 31%] tests/docs/test_mkdocs_links.py::test_nav_files_exist PASSED [ 31%] tests/evaluate/test_auto_evaluation.py::test_semantic_f1_returns_prediction_without_trace PASSED [ 31%] tests/evaluate/test_auto_evaluation.py::test_semantic_f1_returns_prediction_with_trace PASSED [ 31%] tests/evaluate/test_auto_evaluation.py::test_semantic_f1_score_value PASSED [ 31%] tests/evaluate/test_auto_evaluation.py::test_complete_and_grounded_returns_prediction_without_trace PASSED [ 31%] tests/evaluate/test_auto_evaluation.py::test_complete_and_grounded_returns_prediction_with_trace PASSED [ 31%] tests/evaluate/test_auto_evaluation.py::test_semantic_f1_prediction_can_be_compared PASSED [ 31%] tests/evaluate/test_evaluate.py::test_evaluate_initialization PASSED [ 32%] tests/evaluate/test_evaluate.py::test_evaluate_call PASSED [ 32%] tests/evaluate/test_evaluate.py::test_evaluate_single_thread_runs_in_main_thread PASSED [ 32%] tests/evaluate/test_evaluate.py::test_construct_result_df SKIPPED (need --extra option to run) [ 32%] tests/evaluate/test_evaluate.py::test_multithread_evaluate_call PASSED [ 32%] tests/evaluate/test_evaluate.py::test_multi_thread_evaluate_call_cancelled PASSED [ 32%] tests/evaluate/test_evaluate.py::test_evaluate_call_wrong_answer PASSED [ 32%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example0] SKIPPED [ 32%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example1] SKIPPED [ 32%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example2] SKIPPED [ 32%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example3] SKIPPED [ 32%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example0] SKIPPED [ 32%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example1] SKIPPED [ 32%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example2] SKIPPED [ 33%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example3] SKIPPED [ 33%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example0] SKIPPED [ 33%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example1] SKIPPED [ 33%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example2] SKIPPED [ 33%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example3] SKIPPED [ 33%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example0] SKIPPED [ 33%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example1] SKIPPED [ 33%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example2] SKIPPED [ 33%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example3] SKIPPED [ 33%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example0] SKIPPED [ 33%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example1] SKIPPED [ 33%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example2] SKIPPED [ 33%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example3] SKIPPED [ 34%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example0] SKIPPED [ 34%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example1] SKIPPED [ 34%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example2] SKIPPED [ 34%] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example3] SKIPPED [ 34%] tests/evaluate/test_evaluate.py::test_evaluate_callback PASSED [ 34%] tests/evaluate/test_evaluate.py::test_evaluation_result_repr PASSED [ 34%] tests/evaluate/test_evaluate.py::test_evaluate_save_as_json_with_history PASSED [ 34%] tests/evaluate/test_evaluate.py::test_evaluate_save_as_csv_with_history PASSED [ 34%] tests/evaluate/test_metrics.py::test_answer_exact_match_string PASSED [ 34%] tests/evaluate/test_metrics.py::test_answer_exact_match_list PASSED [ 34%] tests/evaluate/test_metrics.py::test_answer_exact_match_no_match PASSED [ 34%] tests/metadata/test_metadata.py::test_metadata PASSED [ 34%] tests/predict/test_aggregation.py::test_majority_with_prediction PASSED [ 35%] tests/predict/test_aggregation.py::test_majority_with_completions PASSED [ 35%] tests/predict/test_aggregation.py::test_majority_with_list PASSED [ 35%] tests/predict/test_aggregation.py::test_majority_with_normalize PASSED [ 35%] tests/predict/test_aggregation.py::test_majority_with_field PASSED [ 35%] tests/predict/test_aggregation.py::test_majority_with_no_majority PASSED [ 35%] tests/predict/test_best_of_n.py::test_refine_forward_success_first_attempt PASSED [ 35%] tests/predict/test_best_of_n.py::test_refine_module_default_fail_count PASSED [ 35%] tests/predict/test_best_of_n.py::test_refine_module_custom_fail_count PASSED [ 35%] tests/predict/test_chain_of_thought.py::test_initialization_with_string_signature PASSED [ 35%] tests/predict/test_chain_of_thought.py::test_async_chain_of_thought PASSED [ 35%] tests/predict/test_chain_of_thought.py::test_chain_of_thought_with_native_reasoning PASSED [ 35%] tests/predict/test_chain_of_thought.py::test_chain_of_thought_with_manual_reasoning PASSED [ 35%] tests/predict/test_code_act.py::test_codeact_code_generation SKIPPED [ 36%] tests/predict/test_code_act.py::test_codeact_support_multiple_fields SKIPPED [ 36%] tests/predict/test_code_act.py::test_codeact_code_parse_failure SKIPPED [ 36%] tests/predict/test_code_act.py::test_codeact_code_execution_failure SKIPPED [ 36%] tests/predict/test_code_act.py::test_codeact_tool_validation SKIPPED [ 36%] tests/predict/test_knn.py::test_knn_initialization SKIPPED (need --extra option to run) [ 36%] tests/predict/test_knn.py::test_knn_query SKIPPED (need --extra option to run) [ 36%] tests/predict/test_knn.py::test_knn_query_specificity SKIPPED (need --extra option to run) [ 36%] tests/predict/test_multi_chain_comparison.py::test_basic_example PASSED [ 36%] tests/predict/test_parallel.py::test_parallel_module PASSED [ 36%] tests/predict/test_parallel.py::test_batch_module PASSED [ 36%] tests/predict/test_parallel.py::test_nested_parallel_module PASSED [ 36%] tests/predict/test_parallel.py::test_nested_batch_method PASSED [ 36%] tests/predict/test_parallel.py::test_batch_with_failed_examples PASSED [ 36%] tests/predict/test_parallel.py::test_parallel_timeout_and_straggler_limit_params PASSED [ 37%] tests/predict/test_parallel.py::test_batch_timeout_and_straggler_limit_params PASSED [ 37%] tests/predict/test_predict.py::test_initialization_with_string_signature PASSED [ 37%] tests/predict/test_predict.py::test_reset_method PASSED [ 37%] tests/predict/test_predict.py::test_lm_after_dump_and_load_state PASSED [ 37%] tests/predict/test_predict.py::test_base_lm_dump_state_ignores_internal_class_marker_kwarg PASSED [ 37%] tests/predict/test_predict.py::test_legacy_lm_state_without_class_marker_loads_as_lm PASSED [ 37%] tests/predict/test_predict.py::test_custom_lm_load_state_requires_trusted_opt_in PASSED [ 37%] tests/predict/test_predict.py::test_nested_custom_lm_class_path_loads_for_trusted_state PASSED [ 37%] tests/predict/test_predict.py::test_call_method PASSED [ 37%] tests/predict/test_predict.py::test_instructions_after_dump_and_load_state PASSED [ 37%] tests/predict/test_predict.py::test_demos_after_dump_and_load_state PASSED [ 37%] tests/predict/test_predict.py::test_typed_demos_after_dump_and_load_state PASSED [ 37%] tests/predict/test_predict.py::test_signature_fields_after_dump_and_load_state PASSED [ 38%] tests/predict/test_predict.py::test_lm_field_after_dump_and_load_state[model.json] PASSED [ 38%] tests/predict/test_predict.py::test_lm_field_after_dump_and_load_state[model.pkl] PASSED [ 38%] tests/predict/test_predict.py::test_load_ignores_serialized_endpoint_override_by_default[api_base] PASSED [ 38%] tests/predict/test_predict.py::test_load_ignores_serialized_endpoint_override_by_default[base_url] PASSED [ 38%] tests/predict/test_predict.py::test_load_allows_serialized_endpoint_override_with_opt_in[api_base] PASSED [ 38%] tests/predict/test_predict.py::test_load_allows_serialized_endpoint_override_with_opt_in[base_url] PASSED [ 38%] tests/predict/test_predict.py::test_load_state_ignores_serialized_endpoint_override_by_default[api_base] PASSED [ 38%] tests/predict/test_predict.py::test_load_state_ignores_serialized_endpoint_override_by_default[base_url] PASSED [ 38%] tests/predict/test_predict.py::test_load_state_allows_serialized_endpoint_override_with_opt_in[api_base] PASSED [ 38%] tests/predict/test_predict.py::test_load_state_allows_serialized_endpoint_override_with_opt_in[base_url] PASSED [ 38%] tests/predict/test_predict.py::test_load_state_ignores_serialized_model_list_endpoint_override_by_default PASSED [ 38%] tests/predict/test_predict.py::test_load_prevents_serialized_endpoint_override_reaching_litellm[api_base] PASSED [ 38%] tests/predict/test_predict.py::test_load_prevents_serialized_endpoint_override_reaching_litellm[base_url] PASSED [ 39%] tests/predict/test_predict.py::test_load_blocks_serialized_model_list_unless_opted_in PASSED [ 39%] tests/predict/test_predict.py::test_load_uses_env_api_key_without_honoring_serialized_endpoint_override PASSED [ 39%] tests/predict/test_predict.py::test_forward_method PASSED [ 39%] tests/predict/test_predict.py::test_forward_method2 PASSED [ 39%] tests/predict/test_predict.py::test_config_management PASSED [ 39%] tests/predict/test_predict.py::test_multi_output PASSED [ 39%] tests/predict/test_predict.py::test_multi_output2 PASSED [ 39%] tests/predict/test_predict.py::test_datetime_inputs_and_outputs PASSED [ 39%] tests/predict/test_predict.py::test_explicitly_valued_enum_inputs_and_outputs PASSED [ 39%] tests/predict/test_predict.py::test_enum_inputs_and_outputs_with_shared_names_and_values PASSED [ 39%] tests/predict/test_predict.py::test_auto_valued_enum_inputs_and_outputs PASSED [ 39%] tests/predict/test_predict.py::test_named_predictors PASSED [ 39%] tests/predict/test_predict.py::test_output_only PASSED [ 40%] tests/predict/test_predict.py::test_load_state_chaining PASSED [ 40%] tests/predict/test_predict.py::test_call_predict_with_chat_history[chat] PASSED [ 40%] tests/predict/test_predict.py::test_call_predict_with_chat_history[json] PASSED [ 40%] tests/predict/test_predict.py::test_lm_usage PASSED [ 40%] tests/predict/test_predict.py::test_lm_usage_with_parallel PASSED [ 40%] tests/predict/test_predict.py::test_lm_usage_with_async PASSED [ 40%] tests/predict/test_predict.py::test_positional_arguments PASSED [ 40%] tests/predict/test_predict.py::test_error_message_on_invalid_lm_setup PASSED [ 40%] tests/predict/test_predict.py::test_field_constraints[chat] PASSED [ 40%] tests/predict/test_predict.py::test_field_constraints[json] PASSED [ 40%] tests/predict/test_predict.py::test_async_predict PASSED [ 40%] tests/predict/test_predict.py::test_predicted_outputs_piped_from_predict_to_lm_call PASSED [ 40%] tests/predict/test_predict.py::test_dump_state_pydantic_non_primitive_types PASSED [ 40%] tests/predict/test_predict.py::test_trace_size_limit PASSED [ 41%] tests/predict/test_predict.py::test_disable_trace PASSED [ 41%] tests/predict/test_predict.py::test_per_module_history_size_limit PASSED [ 41%] tests/predict/test_predict.py::test_per_module_history_disabled PASSED [ 41%] tests/predict/test_predict.py::test_input_field_default_value PASSED [ 41%] tests/predict/test_predict.py::test_extra_fields_warning PASSED [ 41%] tests/predict/test_predict.py::test_missing_optional_input_field_no_warning PASSED [ 41%] tests/predict/test_predict.py::test_missing_required_input_field_still_warns PASSED [ 41%] tests/predict/test_predict.py::test_warning_images PASSED [ 41%] tests/predict/test_predict.py::test_type_mismatch_warning PASSED [ 41%] tests/predict/test_predict.py::test_correct_types_no_warning PASSED [ 41%] tests/predict/test_predict.py::test_list_type_validation PASSED [ 41%] tests/predict/test_predict.py::test_literal_type_validation PASSED [ 41%] tests/predict/test_predict.py::test_literal_union_type_validation PASSED [ 42%] tests/predict/test_predict.py::test_list_string PASSED [ 42%] tests/predict/test_predict.py::test_nested_list_type_validation PASSED [ 42%] tests/predict/test_predict.py::test_nested_dict_type_validation PASSED [ 42%] tests/predict/test_predict.py::test_nested_tuple_type_validation PASSED [ 42%] tests/predict/test_predict.py::test_literal_type_validation_string_signature PASSED [ 42%] tests/predict/test_predict.py::test_list_type_validation_string_signature PASSED [ 42%] tests/predict/test_predict.py::test_dict_type_validation_string_signature PASSED [ 42%] tests/predict/test_predict.py::test_tuple_type_validation_string_signature PASSED [ 42%] tests/predict/test_predict.py::test_union_type_validation_string_signature PASSED [ 42%] tests/predict/test_predict.py::test_basic_types_string_signature[False] PASSED [ 42%] tests/predict/test_predict.py::test_basic_types_string_signature[True] PASSED [ 42%] tests/predict/test_predict.py::test_untyped_string_signature PASSED [ 42%] tests/predict/test_predict.py::test_untyped_class_signature PASSED [ 43%] tests/predict/test_predict.py::test_string_to_list_signature PASSED [ 43%] tests/predict/test_predict.py::test_custom_signature_types[False] PASSED [ 43%] tests/predict/test_predict.py::test_custom_signature_types[True] PASSED [ 43%] tests/predict/test_program_of_thought.py::test_pot_code_generation SKIPPED [ 43%] tests/predict/test_program_of_thought.py::test_old_style_pot SKIPPED [ 43%] tests/predict/test_program_of_thought.py::test_pot_support_multiple_fields SKIPPED [ 43%] tests/predict/test_program_of_thought.py::test_pot_code_generation_with_one_error SKIPPED [ 43%] tests/predict/test_program_of_thought.py::test_pot_code_generation_persistent_errors SKIPPED [ 43%] tests/predict/test_program_of_thought.py::test_pot_code_parse_error PASSED [ 43%] tests/predict/test_react.py::test_tool_observation_preserves_custom_type SKIPPED [ 43%] tests/predict/test_react.py::test_tool_calling_with_pydantic_args PASSED [ 43%] tests/predict/test_react.py::test_react_with_tools_skips_native_response_issubclass_for_generic_alias PASSED [ 43%] tests/predict/test_react.py::test_tool_calling_without_typehint PASSED [ 44%] tests/predict/test_react.py::test_trajectory_truncation PASSED [ 44%] tests/predict/test_react.py::test_context_window_exceeded_after_retries PASSED [ 44%] tests/predict/test_react.py::test_error_retry PASSED [ 44%] tests/predict/test_react.py::test_async_tool_calling_with_pydantic_args PASSED [ 44%] tests/predict/test_react.py::test_async_error_retry PASSED [ 44%] tests/predict/test_react_v2.py::test_react_v2_submit_tool_returns_original_output_fields PASSED [ 44%] tests/predict/test_react_v2.py::test_react_v2_text_mock_lm_loop_records_inputs_once PASSED [ 44%] tests/predict/test_react_v2.py::test_react_v2_continuation_omits_missing_original_inputs PASSED [ 44%] tests/predict/test_react_v2.py::test_react_v2_text_mode_accepts_top_level_tool_arguments PASSED [ 44%] tests/predict/test_react_v2.py::test_react_v2_text_mode_accepts_wrapped_submit_arguments PASSED [ 44%] tests/predict/test_react_v2.py::test_react_v2_unknown_tool_observation_can_continue PASSED [ 44%] tests/predict/test_react_v2.py::test_react_v2_accepts_serialized_history_input PASSED [ 44%] tests/predict/test_react_v2.py::test_react_v2_forced_submit_on_empty_tool_calls PASSED [ 45%] tests/predict/test_react_v2.py::test_react_v2_native_tool_loop_replays_tool_result_with_provider_id PASSED [ 45%] tests/predict/test_react_v2.py::test_react_v2_native_parallel_tool_calls_are_requested_and_replayed PASSED [ 45%] tests/predict/test_refine.py::test_refine_forward_success_first_attempt PASSED [ 45%] tests/predict/test_refine.py::test_refine_module_default_fail_count PASSED [ 45%] tests/predict/test_refine.py::test_refine_module_custom_fail_count PASSED [ 45%] tests/predict/test_rlm.py::TestMockInterpreter::test_scripted_responses PASSED [ 45%] tests/predict/test_rlm.py::TestMockInterpreter::test_returns_final_output_result PASSED [ 45%] tests/predict/test_rlm.py::TestMockInterpreter::test_raises_exception_from_responses PASSED [ 45%] tests/predict/test_rlm.py::TestMockInterpreter::test_records_call_history PASSED [ 45%] tests/predict/test_rlm.py::TestRLMInitialization::test_basic_initialization PASSED [ 45%] tests/predict/test_rlm.py::TestRLMInitialization::test_custom_signature PASSED [ 45%] tests/predict/test_rlm.py::TestRLMInitialization::test_custom_tools PASSED [ 45%] tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_invalid_identifier[invalid-name] PASSED [ 45%] tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_invalid_identifier[123start] PASSED [ 46%] tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_reserved_names[llm_query] PASSED [ 46%] tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_reserved_names[SUBMIT] PASSED [ 46%] tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_reserved_names[print] PASSED [ 46%] tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_not_callable[not a function] PASSED [ 46%] tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_not_callable[123] PASSED [ 46%] tests/predict/test_rlm.py::TestRLMInitialization::test_tools_dict_rejected PASSED [ 46%] tests/predict/test_rlm.py::TestRLMInitialization::test_optional_parameters PASSED [ 46%] tests/predict/test_rlm.py::TestRLMInitialization::test_forward_validates_required_inputs PASSED [ 46%] tests/predict/test_rlm.py::TestRLMInitialization::test_batched_query_errors_have_clear_markers PASSED [ 46%] tests/predict/test_rlm.py::TestRLMInitialization::test_tools_call_counter_is_thread_safe PASSED [ 46%] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```python\nprint(1)\n```-print(1)] PASSED [ 46%] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```py\nx = 1\nprint(x)\n```-x = 1\nprint(x)] PASSED [ 46%] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```\nprint('no lang')\n```-print('no lang')] PASSED [ 47%] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[not fenced code-not fenced code] PASSED [ 47%] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[I'll inspect first.\n```python\nprint('hello')\n```\nThen I will submit.-print('hello')] PASSED [ 47%] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```python\nprint(1)\n```\nsome trailing text-print(1)] PASSED [ 47%] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```python\nprint('oops')-print('oops')] PASSED [ 47%] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```\n```python\nprint(1)\n```\n```-print(1)] PASSED [ 47%] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```\n```\nprint(2)\n```\n```-print(2)] PASSED [ 47%] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences_rejects_non_python_lang PASSED [ 47%] tests/predict/test_rlm.py::TestRLMFormatting::test_format_history PASSED [ 47%] tests/predict/test_rlm.py::TestRLMFormatting::test_format_history_empty PASSED [ 47%] tests/predict/test_rlm.py::TestRLMFormatting::test_action_signature_has_iteration_field PASSED [ 47%] tests/predict/test_rlm.py::TestRLMFormatting::test_format_output PASSED [ 47%] tests/predict/test_rlm.py::TestRLMFormatting::test_format_output_empty PASSED [ 47%] tests/predict/test_rlm.py::TestRLMFormatting::test_format_output_passthrough PASSED [ 48%] tests/predict/test_rlm.py::TestRLMFormatting::test_format_variable_info_string PASSED [ 48%] tests/predict/test_rlm.py::TestRLMFormatting::test_format_variable_info_dict PASSED [ 48%] tests/predict/test_rlm.py::TestRLMFormatting::test_build_variables_multiple PASSED [ 48%] tests/predict/test_rlm.py::TestREPLTypes::test_repl_history_immutability PASSED [ 48%] tests/predict/test_rlm.py::TestREPLTypes::test_repl_history_len_iter_bool PASSED [ 48%] tests/predict/test_rlm.py::TestREPLTypes::test_repl_entry_format PASSED [ 48%] tests/predict/test_rlm.py::TestREPLTypes::test_repl_entry_format_truncation PASSED [ 48%] tests/predict/test_rlm.py::TestREPLTypes::test_repl_entry_format_no_truncation PASSED [ 48%] tests/predict/test_rlm.py::TestREPLTypes::test_repl_history_threads_max_output_chars PASSED [ 48%] tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_from_value PASSED [ 48%] tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_truncation PASSED [ 48%] tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_with_field_info PASSED [ 48%] tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_without_field_info PASSED [ 49%] tests/predict/test_rlm.py::TestREPLTypes::test_build_variables_includes_field_metadata PASSED [ 49%] tests/predict/test_rlm.py::TestRLMCallMethod::test_call_is_alias_for_forward PASSED [ 49%] tests/predict/test_rlm.py::TestRLMMaxIterationsFallback::test_max_iters_triggers_extract PASSED [ 49%] tests/predict/test_rlm.py::TestRLMToolExceptions::test_tool_exception_returns_error_in_output PASSED [ 49%] tests/predict/test_rlm.py::TestRLMToolExceptions::test_runtime_error_history_uses_stripped_code PASSED [ 49%] tests/predict/test_rlm.py::TestRLMToolExceptions::test_syntax_error_from_execute_is_recoverable PASSED [ 49%] tests/predict/test_rlm.py::TestRLMToolExceptions::test_syntax_error_from_strip_code_fences_is_recoverable PASSED [ 49%] tests/predict/test_rlm.py::TestRLMDynamicSignature::test_action_signature_structure PASSED [ 49%] tests/predict/test_rlm.py::TestRLMDynamicSignature::test_extract_signature_structure PASSED [ 49%] tests/predict/test_rlm.py::TestPythonInterpreter::test_start_prewarms_sandbox SKIPPED [ 49%] tests/predict/test_rlm.py::TestPythonInterpreter::test_start_is_idempotent SKIPPED [ 49%] tests/predict/test_rlm.py::TestPythonInterpreter::test_basic_execution SKIPPED [ 49%] tests/predict/test_rlm.py::TestPythonInterpreter::test_variable_injection SKIPPED [ 50%] tests/predict/test_rlm.py::TestPythonInterpreter::test_variable_injection_with_none_values SKIPPED [ 50%] tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_call_kwargs SKIPPED [ 50%] tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_call_positional SKIPPED [ 50%] tests/predict/test_rlm.py::TestPythonInterpreter::test_multiple_tools SKIPPED [ 50%] tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_returns_list SKIPPED [ 50%] tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_returns_dict SKIPPED [ 50%] tests/predict/test_rlm.py::TestPythonInterpreter::test_state_persists SKIPPED [ 50%] tests/predict/test_rlm.py::TestPythonInterpreter::test_syntax_error SKIPPED [ 50%] tests/predict/test_rlm.py::TestPythonInterpreter::test_runtime_error SKIPPED [ 50%] tests/predict/test_rlm.py::TestSandboxSecurity::test_no_network_access SKIPPED [ 50%] tests/predict/test_rlm.py::TestSandboxSecurity::test_imports_work SKIPPED [ 50%] tests/predict/test_rlm.py::TestRLMAsyncMock::test_aforward_basic PASSED [ 50%] tests/predict/test_rlm.py::TestRLMAsyncMock::test_aforward_int_output_mock PASSED [ 50%] tests/predict/test_rlm.py::TestRLMAsyncMock::test_aforward_multi_iteration_mock PASSED [ 51%] tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[count-int-42-SUBMIT(42)-42] PASSED [ 51%] tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[score-float-3.14-SUBMIT(3.14)-3.14] PASSED [ 51%] tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[valid-bool-True-SUBMIT(True)-True] PASSED [ 51%] tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[numbers-list[int]-final_value3-SUBMIT([1, 2, 3])-expected3] PASSED [ 51%] tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[answer-Literal['yes', 'no']-yes-SUBMIT("yes")-yes] PASSED [ 51%] tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_error_retries PASSED [ 51%] tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[count-int-SUBMIT(42)-42-int] SKIPPED [ 51%] tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[score-float-SUBMIT(3.14)-3.14-float] SKIPPED [ 51%] tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[valid-bool-SUBMIT(True)-True-bool] SKIPPED [ 51%] tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[numbers-list[int]-SUBMIT([1, 2, 3])-expected3-list] SKIPPED [ 51%] tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[data-dict[str, str]-SUBMIT({"key": "value"})-expected4-dict] SKIPPED [ 51%] tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[answer-Literal['yes', 'no']-SUBMIT("yes")-yes-str] SKIPPED [ 51%] tests/predict/test_rlm.py::TestRLMTypeCoercion::test_submit_extracts_typed_value SKIPPED [ 52%] tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_final_kwargs SKIPPED [ 52%] tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_final_positional SKIPPED [ 52%] tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_three_fields SKIPPED [ 52%] tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_final_missing_field_errors SKIPPED [ 52%] tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_submit_vars SKIPPED [ 52%] tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_type_coercion SKIPPED [ 52%] tests/predict/test_rlm.py::TestRLMWithDummyLM::test_simple_computation_e2e SKIPPED [ 52%] tests/predict/test_rlm.py::TestRLMWithDummyLM::test_multi_turn_computation_e2e SKIPPED [ 52%] tests/predict/test_rlm.py::TestRLMWithDummyLM::test_with_input_variables_e2e SKIPPED [ 52%] tests/predict/test_rlm.py::TestRLMWithDummyLM::test_with_tool_e2e SKIPPED [ 52%] tests/predict/test_rlm.py::TestRLMWithDummyLM::test_aforward_simple_computation_e2e SKIPPED [ 52%] tests/predict/test_rlm.py::TestRLMWithDummyLM::test_aforward_multi_turn_e2e SKIPPED [ 52%] tests/predict/test_rlm.py::TestRLMWithDummyLM::test_aforward_with_input_variables_e2e SKIPPED [ 53%] tests/predict/test_rlm.py::TestRLMIntegration::test_simple_computation SKIPPED [ 53%] tests/predict/test_rlm.py::TestRLMIntegration::test_with_llm_query SKIPPED [ 53%] tests/predict/test_rlm.py::TestBuildVariablesWithSerializable::test_serializable_uses_build_repl_variable PASSED [ 53%] tests/predict/test_rlm.py::TestBuildVariablesWithSerializable::test_regular_values_unchanged PASSED [ 53%] tests/predict/test_rlm.py::TestPrepareSerializableVars::test_separates_serializable_from_regular PASSED [ 53%] tests/predict/test_rlm.py::TestPrepareSerializableVars::test_no_serializable_returns_all PASSED [ 53%] tests/predict/test_rlm.py::TestPrepareSerializableVars::test_binary_payload_uses_base64_transport PASSED [ 53%] tests/predict/test_rlm.py::TestPrepareSerializableVars::test_large_payload_not_inlined_in_code PASSED [ 53%] tests/predict/test_rlm.py::TestPrepareSerializableVars::test_forward_with_serializable PASSED [ 53%] tests/predict/test_rlm.py::TestLargeSerializableRoundTrip::test_large_payload_round_trips_through_real_sandbox SKIPPED [ 53%] tests/primitives/test_base_module.py::test_deepcopy_basic PASSED [ 53%] tests/primitives/test_base_module.py::test_deepcopy_with_uncopyable_modules PASSED [ 53%] tests/primitives/test_base_module.py::test_deepcopy_with_nested_modules PASSED [ 54%] tests/primitives/test_base_module.py::test_save_and_load_with_json PASSED [ 54%] tests/primitives/test_base_module.py::test_save_and_load_with_pkl SKIPPED [ 54%] tests/primitives/test_base_module.py::test_save_with_extra_modules PASSED [ 54%] tests/primitives/test_base_module.py::test_load_with_version_mismatch PASSED [ 54%] tests/primitives/test_base_module.py::test_single_module_call_with_usage_tracker SKIPPED [ 54%] tests/primitives/test_base_module.py::test_multi_module_call_with_usage_tracker SKIPPED [ 54%] tests/primitives/test_base_module.py::test_usage_tracker_in_parallel SKIPPED [ 54%] tests/primitives/test_base_module.py::test_usage_tracker_async_parallel PASSED [ 54%] tests/primitives/test_base_module.py::test_usage_tracker_no_side_effect PASSED [ 54%] tests/primitives/test_base_module.py::test_module_history PASSED [ 54%] tests/primitives/test_base_module.py::test_module_history_with_concurrency PASSED [ 54%] tests/primitives/test_base_module.py::test_module_history_async PASSED [ 54%] tests/primitives/test_base_module.py::test_forward_direct_call_warning PASSED [ 54%] tests/primitives/test_base_module.py::test_forward_through_call_no_warning PASSED [ 55%] tests/primitives/test_example.py::test_example_initialization PASSED [ 55%] tests/primitives/test_example.py::test_example_initialization_from_base PASSED [ 55%] tests/primitives/test_example.py::test_example_initialization_from_dict PASSED [ 55%] tests/primitives/test_example.py::test_example_set_get_item PASSED [ 55%] tests/primitives/test_example.py::test_example_attribute_access PASSED [ 55%] tests/primitives/test_example.py::test_example_deletion PASSED [ 55%] tests/primitives/test_example.py::test_example_len PASSED [ 55%] tests/primitives/test_example.py::test_example_repr_str_img PASSED [ 55%] tests/primitives/test_example.py::test_example_repr_str PASSED [ 55%] tests/primitives/test_example.py::test_example_eq PASSED [ 55%] tests/primitives/test_example.py::test_example_hash PASSED [ 55%] tests/primitives/test_example.py::test_example_keys_values_items PASSED [ 55%] tests/primitives/test_example.py::test_example_get PASSED [ 56%] tests/primitives/test_example.py::test_example_with_inputs PASSED [ 56%] tests/primitives/test_example.py::test_example_inputs_labels PASSED [ 56%] tests/primitives/test_example.py::test_example_copy_without PASSED [ 56%] tests/primitives/test_example.py::test_example_to_dict PASSED [ 56%] tests/primitives/test_example.py::test_example_to_dict_with_history PASSED [ 56%] tests/primitives/test_module.py::test_module_initialization PASSED [ 56%] tests/primitives/test_module.py::test_named_predictors PASSED [ 56%] tests/primitives/test_module.py::test_predictors PASSED [ 56%] tests/primitives/test_module.py::test_forward PASSED [ 56%] tests/primitives/test_module.py::test_nested_named_predictors PASSED [ 56%] tests/primitives/test_module.py::test_empty_module PASSED [ 56%] tests/primitives/test_module.py::test_single_level PASSED [ 56%] tests/primitives/test_module.py::test_multiple_levels PASSED [ 57%] tests/primitives/test_module.py::test_multiple_sub_modules PASSED [ 57%] tests/primitives/test_module.py::test_non_base_module_attributes PASSED [ 57%] tests/primitives/test_module.py::test_complex_module_traversal PASSED [ 57%] tests/primitives/test_module.py::test_complex_module_traversal_with_same_module PASSED [ 57%] tests/primitives/test_module.py::test_complex_module_set_attribute_by_name PASSED [ 57%] tests/primitives/test_module.py::test_named_parameters_duplicate_references PASSED [ 57%] tests/primitives/test_module.py::test_load_dspy_program_cross_version PASSED [ 57%] tests/primitives/test_module.py::test_load_state_is_transactional PASSED [ 57%] tests/primitives/test_python_interpreter.py::test_execute_simple_code SKIPPED [ 57%] tests/primitives/test_python_interpreter.py::test_import SKIPPED (need --deno option to run) [ 57%] tests/primitives/test_python_interpreter.py::test_user_variable_definitions SKIPPED [ 57%] tests/primitives/test_python_interpreter.py::test_rejects_python_keywords_as_variable_names SKIPPED [ 57%] tests/primitives/test_python_interpreter.py::test_failure_syntax_error SKIPPED [ 58%] tests/primitives/test_python_interpreter.py::test_failure_zero_division SKIPPED [ 58%] tests/primitives/test_python_interpreter.py::test_exception_args SKIPPED [ 58%] tests/primitives/test_python_interpreter.py::test_submit_with_list SKIPPED [ 58%] tests/primitives/test_python_interpreter.py::test_enable_env_vars_flag SKIPPED [ 58%] tests/primitives/test_python_interpreter.py::test_read_file_access_control SKIPPED [ 58%] tests/primitives/test_python_interpreter.py::test_enable_write_flag SKIPPED [ 58%] tests/primitives/test_python_interpreter.py::test_enable_net_flag SKIPPED [ 58%] tests/primitives/test_python_interpreter.py::test_interpreter_security_filesystem_access SKIPPED [ 58%] tests/primitives/test_python_interpreter.py::test_tools_dict_is_copied SKIPPED [ 58%] tests/primitives/test_python_interpreter.py::test_serialize_tuple SKIPPED [ 58%] tests/primitives/test_python_interpreter.py::test_serialize_set SKIPPED [ 58%] tests/primitives/test_python_interpreter.py::test_serialize_set_mixed_types SKIPPED [ 58%] tests/primitives/test_python_interpreter.py::test_deno_command_dict_raises_type_error SKIPPED [ 59%] tests/primitives/test_python_interpreter.py::test_tool_with_typed_signature SKIPPED [ 59%] tests/primitives/test_python_interpreter.py::test_tool_positional_args SKIPPED [ 59%] tests/primitives/test_python_interpreter.py::test_tool_keyword_args SKIPPED [ 59%] tests/primitives/test_python_interpreter.py::test_tool_default_args SKIPPED [ 59%] tests/primitives/test_python_interpreter.py::test_tools_re_register_after_process_restart SKIPPED [ 59%] tests/primitives/test_python_interpreter.py::test_mounts_replay_after_process_restart SKIPPED [ 59%] tests/primitives/test_python_interpreter.py::test_tool_all_positional_args SKIPPED [ 59%] tests/primitives/test_python_interpreter.py::test_tool_error_surfaces_as_runtime_error SKIPPED [ 59%] tests/primitives/test_python_interpreter.py::test_tool_async_def_function SKIPPED [ 59%] tests/primitives/test_python_interpreter.py::test_tool_async_def_raises_propagates SKIPPED [ 59%] tests/primitives/test_python_interpreter.py::test_submit_with_typed_signature SKIPPED [ 59%] tests/primitives/test_python_interpreter.py::test_submit_positional_args SKIPPED [ 59%] tests/primitives/test_python_interpreter.py::test_submit_multi_output SKIPPED [ 59%] tests/primitives/test_python_interpreter.py::test_submit_wrong_arg_count SKIPPED [ 60%] tests/primitives/test_python_interpreter.py::test_extract_parameters SKIPPED [ 60%] tests/primitives/test_python_interpreter.py::test_extract_parameters_complex_types SKIPPED [ 60%] tests/primitives/test_python_interpreter.py::test_large_variable_injection SKIPPED [ 60%] tests/primitives/test_python_interpreter.py::test_large_variable_content_integrity SKIPPED [ 60%] tests/primitives/test_python_interpreter.py::test_mixed_small_and_large_variables SKIPPED [ 60%] tests/primitives/test_python_interpreter.py::test_multiple_large_variables SKIPPED [ 60%] tests/primitives/test_python_interpreter.py::test_large_list_variable SKIPPED [ 60%] tests/primitives/test_python_interpreter.py::test_nested_sets_and_tuples SKIPPED [ 60%] tests/primitives/test_python_interpreter.py::test_small_variable_not_using_filesystem SKIPPED [ 60%] tests/primitives/test_python_interpreter.py::test_large_variable_threshold_boundary SKIPPED [ 60%] tests/primitives/test_python_interpreter.py::test_enable_read_paths_symlink SKIPPED [ 60%] tests/primitives/test_python_interpreter.py::test_enable_read_paths_multiple_files SKIPPED [ 60%] tests/primitives/test_sandbox_serializable.py::TestABCConformance::test_subclass_conformance PASSED [ 61%] tests/primitives/test_sandbox_serializable.py::TestABCConformance::test_incomplete_subclass_cannot_instantiate PASSED [ 61%] tests/primitives/test_sandbox_serializable.py::TestABCConformance::test_structural_conformance_no_longer_accepted PASSED [ 61%] tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_sandbox_setup PASSED [ 61%] tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_to_sandbox_returns_bytes PASSED [ 61%] tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_sandbox_assignment PASSED [ 61%] tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_rlm_preview PASSED [ 61%] tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_rlm_preview_truncation PASSED [ 61%] tests/primitives/test_sandbox_serializable.py::TestBuildReplVariable::test_builds_variable_with_preview_from_rlm_preview PASSED [ 61%] tests/primitives/test_sandbox_serializable.py::TestBuildReplVariable::test_surfaces_sandbox_setup_in_description PASSED [ 61%] tests/primitives/test_sandbox_serializable.py::TestBuildReplVariable::test_passes_field_info_through PASSED [ 61%] tests/primitives/test_sandbox_serializable.py::TestToReplVariableMethod::test_default_to_repl_variable PASSED [ 61%] tests/primitives/test_sandbox_serializable.py::TestSignatureAnnotation::test_subclass_supports_signature_annotation PASSED [ 61%] tests/propose/test_grounded_proposer.py::test_propose_instructions_for_program[None] PASSED [ 62%] tests/propose/test_grounded_proposer.py::test_propose_instructions_for_program[demo_candidates1] PASSED [ 62%] tests/propose/test_grounded_proposer.py::test_propose_instruction_for_predictor[None] PASSED [ 62%] tests/propose/test_grounded_proposer.py::test_propose_instruction_for_predictor[demo_candidates1] PASSED [ 62%] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-input_formats-test_markdown_1-input1] SKIPPED [ 62%] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-input_formats-test_markdown_1-input2] SKIPPED [ 62%] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_many_types_1-input1] SKIPPED [ 62%] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_many_types_1-input2] SKIPPED [ 62%] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_nesting_2-input1] SKIPPED [ 62%] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_nesting_1-input1] SKIPPED [ 62%] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_nesting_1-input2] SKIPPED [ 62%] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-input_formats-test_markdown_1-input1] SKIPPED [ 62%] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-input_formats-test_markdown_1-input2] SKIPPED [ 62%] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_many_types_1-input1] SKIPPED [ 63%] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_many_types_1-input2] SKIPPED [ 63%] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_nesting_2-input1] SKIPPED [ 63%] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_nesting_1-input1] SKIPPED [ 63%] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_nesting_1-input2] SKIPPED [ 63%] tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-input_formats-test_markdown_1-input1] SKIPPED [ 63%] tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-input_formats-test_markdown_1-input2] SKIPPED [ 63%] tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_many_types_1-input1] SKIPPED [ 63%] tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_many_types_1-input2] SKIPPED [ 63%] tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_nesting_2-input1] SKIPPED [ 63%] tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_nesting_1-input1] SKIPPED [ 63%] tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_nesting_1-input2] SKIPPED [ 63%] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-input_formats-test_markdown_1-input1] SKIPPED [ 63%] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-input_formats-test_markdown_1-input2] SKIPPED [ 63%] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_many_types_1-input1] SKIPPED [ 64%] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_many_types_1-input2] SKIPPED [ 64%] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_nesting_2-input1] SKIPPED [ 64%] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_nesting_1-input1] SKIPPED [ 64%] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_nesting_1-input2] SKIPPED [ 64%] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-input_formats-test_markdown_1-input1] SKIPPED [ 64%] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-input_formats-test_markdown_1-input2] SKIPPED [ 64%] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_many_types_1-input1] SKIPPED [ 64%] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_many_types_1-input2] SKIPPED [ 64%] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_nesting_2-input1] SKIPPED [ 64%] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_nesting_1-input1] SKIPPED [ 64%] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_nesting_1-input2] SKIPPED [ 64%] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-input_formats-test_markdown_1-input1] SKIPPED [ 64%] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-input_formats-test_markdown_1-input2] SKIPPED [ 65%] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_many_types_1-input1] SKIPPED [ 65%] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_many_types_1-input2] SKIPPED [ 65%] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_nesting_2-input1] SKIPPED [ 65%] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_nesting_1-input1] SKIPPED [ 65%] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_nesting_1-input2] SKIPPED [ 65%] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-input_formats-test_markdown_1-input1] SKIPPED [ 65%] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-input_formats-test_markdown_1-input2] SKIPPED [ 65%] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_many_types_1-input1] SKIPPED [ 65%] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_many_types_1-input2] SKIPPED [ 65%] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_nesting_2-input1] SKIPPED [ 65%] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_nesting_1-input1] SKIPPED [ 65%] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_nesting_1-input2] SKIPPED [ 65%] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-input_formats-test_markdown_1-input1] SKIPPED [ 66%] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-input_formats-test_markdown_1-input2] SKIPPED [ 66%] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_many_types_1-input1] SKIPPED [ 66%] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_many_types_1-input2] SKIPPED [ 66%] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_nesting_2-input1] SKIPPED [ 66%] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_nesting_1-input1] SKIPPED [ 66%] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_nesting_1-input2] SKIPPED [ 66%] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-input_formats-test_markdown_1-input1] SKIPPED [ 66%] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-input_formats-test_markdown_1-input2] SKIPPED [ 66%] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_many_types_1-input1] SKIPPED [ 66%] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_many_types_1-input2] SKIPPED [ 66%] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_nesting_2-input1] SKIPPED [ 66%] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_nesting_1-input1] SKIPPED [ 66%] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_nesting_1-input2] SKIPPED [ 67%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 67%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 67%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 67%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 67%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 67%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 67%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 67%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 67%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 67%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 67%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 67%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 67%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 68%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 68%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 68%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 68%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 68%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 68%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 68%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 68%] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 68%] tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 68%] tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 68%] tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 68%] tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 68%] tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 68%] tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 69%] tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 69%] tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-input_formats-test_markdown_1-input1] SKIPPED [ 69%] tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-input_formats-test_markdown_1-input2] SKIPPED [ 69%] tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_many_types_1-input1] SKIPPED [ 69%] tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_many_types_1-input2] SKIPPED [ 69%] tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_nesting_2-input1] SKIPPED [ 69%] tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_nesting_1-input1] SKIPPED [ 69%] tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_nesting_1-input2] SKIPPED [ 69%] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-4o] SKIPPED [ 69%] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-4o-mini] SKIPPED [ 69%] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-4-turbo] SKIPPED [ 69%] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-o1-preview] SKIPPED [ 69%] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-o1-mini] SKIPPED [ 70%] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[claude-3.5-sonnet] SKIPPED [ 70%] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[claude-3.5-haiku] SKIPPED [ 70%] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gemini-1.5-pro] SKIPPED [ 70%] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gemini-1.5-flash] SKIPPED [ 70%] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.1-405b-instruct] SKIPPED [ 70%] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.1-70b-instruct] SKIPPED [ 70%] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.1-8b-instruct] SKIPPED [ 70%] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.2-3b-instruct] SKIPPED [ 70%] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[deepseek-r1] SKIPPED [ 70%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-Predict] SKIPPED [ 70%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-ChainOfThought] SKIPPED [ 70%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-mini-Predict] SKIPPED [ 70%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-mini-ChainOfThought] SKIPPED [ 71%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4-turbo-Predict] SKIPPED [ 71%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4-turbo-ChainOfThought] SKIPPED [ 71%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-preview-Predict] SKIPPED [ 71%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-preview-ChainOfThought] SKIPPED [ 71%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-mini-Predict] SKIPPED [ 71%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-mini-ChainOfThought] SKIPPED [ 71%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-sonnet-Predict] SKIPPED [ 71%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-sonnet-ChainOfThought] SKIPPED [ 71%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-haiku-Predict] SKIPPED [ 71%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-haiku-ChainOfThought] SKIPPED [ 71%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-pro-Predict] SKIPPED [ 71%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-pro-ChainOfThought] SKIPPED [ 71%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-flash-Predict] SKIPPED [ 72%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-flash-ChainOfThought] SKIPPED [ 72%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-405b-instruct-Predict] SKIPPED [ 72%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-405b-instruct-ChainOfThought] SKIPPED [ 72%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-70b-instruct-Predict] SKIPPED [ 72%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-70b-instruct-ChainOfThought] SKIPPED [ 72%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-8b-instruct-Predict] SKIPPED [ 72%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-8b-instruct-ChainOfThought] SKIPPED [ 72%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.2-3b-instruct-Predict] SKIPPED [ 72%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.2-3b-instruct-ChainOfThought] SKIPPED [ 72%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[deepseek-r1-Predict] SKIPPED [ 72%] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[deepseek-r1-ChainOfThought] SKIPPED [ 72%] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-4o] SKIPPED [ 72%] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-4o-mini] SKIPPED [ 72%] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-4-turbo] SKIPPED [ 73%] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-o1-preview] SKIPPED [ 73%] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-o1-mini] SKIPPED [ 73%] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[claude-3.5-sonnet] SKIPPED [ 73%] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[claude-3.5-haiku] SKIPPED [ 73%] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gemini-1.5-pro] SKIPPED [ 73%] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gemini-1.5-flash] SKIPPED [ 73%] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.1-405b-instruct] SKIPPED [ 73%] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.1-70b-instruct] SKIPPED [ 73%] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.1-8b-instruct] SKIPPED [ 73%] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.2-3b-instruct] SKIPPED [ 73%] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[deepseek-r1] SKIPPED [ 73%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-Predict] SKIPPED [ 73%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-ChainOfThought] SKIPPED [ 74%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-mini-Predict] SKIPPED [ 74%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-mini-ChainOfThought] SKIPPED [ 74%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4-turbo-Predict] SKIPPED [ 74%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4-turbo-ChainOfThought] SKIPPED [ 74%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-preview-Predict] SKIPPED [ 74%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-preview-ChainOfThought] SKIPPED [ 74%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-mini-Predict] SKIPPED [ 74%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-mini-ChainOfThought] SKIPPED [ 74%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-sonnet-Predict] SKIPPED [ 74%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-sonnet-ChainOfThought] SKIPPED [ 74%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-haiku-Predict] SKIPPED [ 74%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-haiku-ChainOfThought] SKIPPED [ 74%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-pro-Predict] SKIPPED [ 75%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-pro-ChainOfThought] SKIPPED [ 75%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-flash-Predict] SKIPPED [ 75%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-flash-ChainOfThought] SKIPPED [ 75%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-405b-instruct-Predict] SKIPPED [ 75%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-405b-instruct-ChainOfThought] SKIPPED [ 75%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-70b-instruct-Predict] SKIPPED [ 75%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-70b-instruct-ChainOfThought] SKIPPED [ 75%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-8b-instruct-Predict] SKIPPED [ 75%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-8b-instruct-ChainOfThought] SKIPPED [ 75%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.2-3b-instruct-Predict] SKIPPED [ 75%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.2-3b-instruct-ChainOfThought] SKIPPED [ 75%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[deepseek-r1-Predict] SKIPPED [ 75%] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[deepseek-r1-ChainOfThought] SKIPPED [ 76%] tests/retrievers/test_colbertv2.py::test_get_request_raises_on_server_error PASSED [ 76%] tests/retrievers/test_colbertv2.py::test_post_request_raises_on_server_error PASSED [ 76%] tests/retrievers/test_colbertv2.py::test_get_request_success PASSED [ 76%] tests/retrievers/test_colbertv2.py::test_post_request_success PASSED [ 76%] tests/retrievers/test_embeddings.py::test_embeddings_basic_search PASSED [ 76%] tests/retrievers/test_embeddings.py::test_embeddings_multithreaded_search PASSED [ 76%] tests/retrievers/test_embeddings.py::test_embeddings_save_load PASSED [ 76%] tests/retrievers/test_embeddings.py::test_embeddings_from_saved PASSED [ 76%] tests/retrievers/test_embeddings.py::test_embeddings_load_nonexistent_path PASSED [ 76%] tests/retrievers/test_embeddings.py::test_embeddings_with_scores_basic_search PASSED [ 76%] tests/retrievers/test_embeddings.py::test_embeddings_with_scores_save_load PASSED [ 76%] tests/signatures/test_adapter_file.py::test_file_from_local_path PASSED [ 76%] tests/signatures/test_adapter_file.py::test_file_from_path_method PASSED [ 77%] tests/signatures/test_adapter_file.py::test_file_from_path_with_custom_filename PASSED [ 77%] tests/signatures/test_adapter_file.py::test_file_from_bytes PASSED [ 77%] tests/signatures/test_adapter_file.py::test_file_from_bytes_with_filename PASSED [ 77%] tests/signatures/test_adapter_file.py::test_file_from_file_id PASSED [ 77%] tests/signatures/test_adapter_file.py::test_file_from_file_id_with_filename PASSED [ 77%] tests/signatures/test_adapter_file.py::test_file_from_dict_with_file_data PASSED [ 77%] tests/signatures/test_adapter_file.py::test_file_from_dict_with_file_id PASSED [ 77%] tests/signatures/test_adapter_file.py::test_file_format_with_file_data PASSED [ 77%] tests/signatures/test_adapter_file.py::test_file_format_with_file_id PASSED [ 77%] tests/signatures/test_adapter_file.py::test_file_repr_with_file_data PASSED [ 77%] tests/signatures/test_adapter_file.py::test_file_repr_with_file_id PASSED [ 77%] tests/signatures/test_adapter_file.py::test_file_str PASSED [ 77%] tests/signatures/test_adapter_file.py::test_encode_file_to_dict_from_path PASSED [ 77%] tests/signatures/test_adapter_file.py::test_encode_file_to_dict_from_bytes PASSED [ 78%] tests/signatures/test_adapter_file.py::test_invalid_file_string PASSED [ 78%] tests/signatures/test_adapter_file.py::test_invalid_dict PASSED [ 78%] tests/signatures/test_adapter_file.py::test_file_in_signature PASSED [ 78%] tests/signatures/test_adapter_file.py::test_file_list_in_signature PASSED [ 78%] tests/signatures/test_adapter_file.py::test_optional_file_field PASSED [ 78%] tests/signatures/test_adapter_file.py::test_save_load_file_signature PASSED [ 78%] tests/signatures/test_adapter_file.py::test_file_frozen PASSED [ 78%] tests/signatures/test_adapter_file.py::test_file_with_all_fields PASSED [ 78%] tests/signatures/test_adapter_file.py::test_file_path_not_found PASSED [ 78%] tests/signatures/test_adapter_file.py::test_file_custom_mime_type PASSED [ 78%] tests/signatures/test_adapter_file.py::test_file_from_bytes_custom_mime PASSED [ 78%] tests/signatures/test_adapter_file.py::test_file_data_uri_in_format PASSED [ 78%] tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case0] PASSED [ 79%] tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case1] PASSED [ 79%] tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case2] PASSED [ 79%] tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case3] PASSED [ 79%] tests/signatures/test_adapter_image.py::test_image_input_formats[pil_image-PIL Image] XFAIL [ 79%] tests/signatures/test_adapter_image.py::test_image_input_formats[encoded_pil_image-encoded PIL image string] XFAILon) [ 79%] tests/signatures/test_adapter_image.py::test_image_input_formats[dspy_image_download-dspy.Image with download=True] PASSED [ 79%] tests/signatures/test_adapter_image.py::test_image_input_formats[dspy_image_no_download-dspy.Image without download] PASSED [ 79%] tests/signatures/test_adapter_image.py::test_predictor_save_load PASSED [ 79%] tests/signatures/test_adapter_image.py::test_save_load_complex_default_types PASSED [ 79%] tests/signatures/test_adapter_image.py::test_save_load_complex_types[test_case0] PASSED [ 79%] tests/signatures/test_adapter_image.py::test_save_load_complex_types[test_case1] PASSED [ 79%] tests/signatures/test_adapter_image.py::test_save_load_pydantic_model PASSED [ 79%] tests/signatures/test_adapter_image.py::test_optional_image_field PASSED [ 80%] tests/signatures/test_adapter_image.py::test_pdf_url_support PASSED [ 80%] tests/signatures/test_adapter_image.py::test_different_mime_types PASSED [ 80%] tests/signatures/test_adapter_image.py::test_mime_type_from_response_headers PASSED [ 80%] tests/signatures/test_adapter_image.py::test_pdf_from_file PASSED [ 80%] tests/signatures/test_adapter_image.py::test_image_repr PASSED [ 80%] tests/signatures/test_adapter_image.py::test_from_methods_warn PASSED [ 80%] tests/signatures/test_adapter_image.py::test_invalid_string_format PASSED [ 80%] tests/signatures/test_adapter_image.py::test_pil_image_with_download_parameter PASSED [ 80%] tests/signatures/test_custom_types.py::test_basic_custom_type_resolution PASSED [ 80%] tests/signatures/test_custom_types.py::test_type_alias_for_nested_types PASSED [ 80%] tests/signatures/test_custom_types.py::test_module_level_type_resolution PASSED [ 80%] tests/signatures/test_custom_types.py::test_recommended_patterns PASSED [ 80%] tests/signatures/test_custom_types.py::test_expected_failure PASSED [ 81%] tests/signatures/test_custom_types.py::test_module_type_resolution PASSED [ 81%] tests/signatures/test_signature.py::test_field_types_and_custom_attributes PASSED [ 81%] tests/signatures/test_signature.py::test_no_input_output PASSED [ 81%] tests/signatures/test_signature.py::test_no_input_output2 PASSED [ 81%] tests/signatures/test_signature.py::test_all_fields_have_prefix PASSED [ 81%] tests/signatures/test_signature.py::test_signature_parsing PASSED [ 81%] tests/signatures/test_signature.py::test_duplicate_input_output_field_names_raise PASSED [ 81%] tests/signatures/test_signature.py::test_with_signature PASSED [ 81%] tests/signatures/test_signature.py::test_append_instructions PASSED [ 81%] tests/signatures/test_signature.py::test_append_instructions_chains PASSED [ 81%] tests/signatures/test_signature.py::test_append_instructions_preserves_default_instructions PASSED [ 81%] tests/signatures/test_signature.py::test_with_updated_field PASSED [ 81%] tests/signatures/test_signature.py::test_empty_signature PASSED [ 81%] tests/signatures/test_signature.py::test_instructions_signature PASSED [ 82%] tests/signatures/test_signature.py::test_signature_instructions PASSED [ 82%] tests/signatures/test_signature.py::test_signature_instructions_none PASSED [ 82%] tests/signatures/test_signature.py::test_signature_from_dict PASSED [ 82%] tests/signatures/test_signature.py::test_signature_equality PASSED [ 82%] tests/signatures/test_signature.py::test_signature_inequality PASSED [ 82%] tests/signatures/test_signature.py::test_equality_format PASSED [ 82%] tests/signatures/test_signature.py::test_signature_reverse PASSED [ 82%] tests/signatures/test_signature.py::test_insert_field_at_various_positions PASSED [ 82%] tests/signatures/test_signature.py::test_order_preserved_with_mixed_annotations PASSED [ 82%] tests/signatures/test_signature.py::test_infer_prefix PASSED [ 82%] tests/signatures/test_signature.py::test_insantiating PASSED [ 82%] tests/signatures/test_signature.py::test_insantiating2 PASSED [ 82%] tests/signatures/test_signature.py::test_multiline_instructions PASSED [ 83%] tests/signatures/test_signature.py::test_dump_and_load_state PASSED [ 83%] tests/signatures/test_signature.py::test_typed_signatures_basic_types PASSED [ 83%] tests/signatures/test_signature.py::test_typed_signatures_generics PASSED [ 83%] tests/signatures/test_signature.py::test_typed_signatures_unions_and_optionals PASSED [ 83%] tests/signatures/test_signature.py::test_typed_signatures_any PASSED [ 83%] tests/signatures/test_signature.py::test_typed_signatures_nested PASSED [ 83%] tests/signatures/test_signature.py::test_typed_signatures_from_dict PASSED [ 83%] tests/signatures/test_signature.py::test_typed_signatures_complex_combinations PASSED [ 83%] tests/signatures/test_signature.py::test_make_signature_from_string PASSED [ 83%] tests/signatures/test_signature.py::test_signature_field_with_constraints PASSED [ 83%] tests/signatures/test_signature.py::test_basic_custom_type PASSED [ 83%] tests/signatures/test_signature.py::test_custom_type_from_different_module PASSED [ 83%] tests/signatures/test_signature.py::test_pep604_union_type_inline PASSED [ 84%] tests/signatures/test_signature.py::test_pep604_union_type_inline_equivalence PASSED [ 84%] tests/signatures/test_signature.py::test_pep604_union_type_inline_nested PASSED [ 84%] tests/signatures/test_signature.py::test_pep604_union_type_class_nested PASSED [ 84%] tests/signatures/test_signature.py::test_pep604_union_type_class_equivalence PASSED [ 84%] tests/signatures/test_signature.py::test_pep604_union_type_insert PASSED [ 84%] tests/signatures/test_signature.py::test_pep604_union_type_with_custom_types PASSED [ 84%] tests/signatures/test_signature.py::test_signature_cloudpickle_roundtrip PASSED [ 84%] tests/signatures/test_signature.py::test_predict_cloudpickle_roundtrip PASSED [ 84%] tests/streaming/test_streaming.py::test_streamify_yields_expected_response_chunks PASSED [ 84%] tests/streaming/test_streaming.py::test_streaming_response_yields_expected_response_chunks PASSED [ 84%] tests/streaming/test_streaming.py::test_default_status_streaming PASSED [ 84%] tests/streaming/test_streaming.py::test_custom_status_streaming PASSED [ 84%] tests/streaming/test_streaming.py::test_concurrent_status_message_providers PASSED [ 85%] tests/streaming/test_streaming.py::test_stream_listener_chat_adapter SKIPPED [ 85%] tests/streaming/test_streaming.py::test_default_status_streaming_in_async_program PASSED [ 85%] tests/streaming/test_streaming.py::test_stream_listener_json_adapter SKIPPED [ 85%] tests/streaming/test_streaming.py::test_streaming_handles_space_correctly PASSED [ 85%] tests/streaming/test_streaming.py::test_sync_streaming SKIPPED (need --llm_call option to run) [ 85%] tests/streaming/test_streaming.py::test_sync_status_streaming PASSED [ 85%] tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_chat_adapter PASSED [ 85%] tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_json_adapter PASSED [ 85%] tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_chat_adapter_untokenized_stream PASSED [ 85%] tests/streaming/test_streaming.py::test_stream_listener_missing_completion_marker_chat_adapter PASSED [ 85%] tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_json_adapter_untokenized_stream PASSED [ 85%] tests/streaming/test_streaming.py::test_status_message_non_blocking PASSED [ 85%] tests/streaming/test_streaming.py::test_status_message_non_blocking_async_program PASSED [ 86%] tests/streaming/test_streaming.py::test_stream_listener_allow_reuse PASSED [ 86%] tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_xml_adapter PASSED [ 86%] tests/streaming/test_streaming.py::test_streaming_allows_custom_chunk_types PASSED [ 86%] tests/streaming/test_streaming.py::test_streaming_allows_custom_streamable_type PASSED [ 86%] tests/streaming/test_streaming.py::test_streaming_with_citations PASSED [ 86%] tests/streaming/test_streaming.py::test_chat_adapter_simple_pydantic_streaming PASSED [ 86%] tests/streaming/test_streaming.py::test_chat_adapter_with_generic_type_annotation PASSED [ 86%] tests/streaming/test_streaming.py::test_chat_adapter_nested_pydantic_streaming PASSED [ 86%] tests/streaming/test_streaming.py::test_chat_adapter_mixed_fields_streaming PASSED [ 86%] tests/streaming/test_streaming.py::test_json_adapter_simple_pydantic_streaming PASSED [ 86%] tests/streaming/test_streaming.py::test_json_adapter_bracket_balance_detection PASSED [ 86%] tests/streaming/test_streaming.py::test_json_adapter_multiple_fields_detection PASSED [ 86%] tests/streaming/test_streaming.py::test_stream_listener_could_form_end_identifier_chat_adapter PASSED [ 86%] tests/streaming/test_streaming.py::test_stream_listener_could_form_end_identifier_json_adapter PASSED [ 87%] tests/streaming/test_streaming.py::test_stream_listener_could_form_end_identifier_xml_adapter PASSED [ 87%] tests/streaming/test_streaming.py::test_streaming_reasoning_model PASSED [ 87%] tests/streaming/test_streaming.py::test_stream_listener_empty_last_chunk_chat_adapter PASSED [ 87%] tests/streaming/test_streaming.py::test_stream_listener_empty_last_chunk_json_adapter PASSED [ 87%] tests/streaming/test_streaming.py::test_streaming_reasoning_fallback PASSED [ 87%] tests/teleprompt/test_bettertogether.py::test_bettertogether_import PASSED [ 87%] tests/teleprompt/test_bettertogether.py::test_bettertogether_initialization_default PASSED [ 87%] tests/teleprompt/test_bettertogether.py::test_bettertogether_initialization_custom PASSED [ 87%] tests/teleprompt/test_bettertogether.py::test_bettertogether_initialization_invalid_optimizer PASSED [ 87%] tests/teleprompt/test_bettertogether.py::test_strategy_validation PASSED [ 87%] tests/teleprompt/test_bettertogether.py::test_compile_basic PASSED [ 87%] tests/teleprompt/test_bettertogether.py::test_trainset_validation PASSED [ 87%] tests/teleprompt/test_bettertogether.py::test_valset_ratio_validation PASSED [ 88%] tests/teleprompt/test_bettertogether.py::test_optimizer_compile_args_validation PASSED [ 88%] tests/teleprompt/test_bettertogether.py::test_student_in_optimizer_compile_args PASSED [ 88%] tests/teleprompt/test_bettertogether.py::test_compile_args_passed_to_optimizer PASSED [ 88%] tests/teleprompt/test_bettertogether.py::test_compile_args_multi_optimizer_strategy PASSED [ 88%] tests/teleprompt/test_bettertogether.py::test_compile_args_override_global_params PASSED [ 88%] tests/teleprompt/test_bettertogether.py::test_trainset_shuffling_between_steps PASSED [ 88%] tests/teleprompt/test_bettertogether.py::test_strategy_execution_order PASSED [ 88%] tests/teleprompt/test_bettertogether.py::test_lm_lifecycle_management PASSED [ 88%] tests/teleprompt/test_bettertogether.py::test_error_handling_returns_best_program PASSED [ 88%] tests/teleprompt/test_bettertogether.py::test_program_selection[test_valset0-p_optimized-With valset: returns best score (p), not latest (w)] PASSED [ 88%] tests/teleprompt/test_bettertogether.py::test_program_selection[None-w_optimized-Without valset: returns latest program (w)] PASSED [ 88%] tests/teleprompt/test_bettertogether.py::test_candidate_programs_structure PASSED [ 88%] tests/teleprompt/test_bettertogether.py::test_empty_valset_handling PASSED [ 89%] tests/teleprompt/test_bootstrap.py::test_bootstrap_initialization PASSED [ 89%] tests/teleprompt/test_bootstrap.py::test_compile_with_predict_instances PASSED [ 89%] tests/teleprompt/test_bootstrap.py::test_bootstrap_effectiveness PASSED [ 89%] tests/teleprompt/test_bootstrap.py::test_error_handling_during_bootstrap PASSED [ 89%] tests/teleprompt/test_bootstrap.py::test_validation_set_usage PASSED [ 89%] tests/teleprompt/test_bootstrap_finetune.py::test_bootstrap_finetune_initialization PASSED [ 89%] tests/teleprompt/test_bootstrap_finetune.py::test_compile_with_predict_instances PASSED [ 89%] tests/teleprompt/test_bootstrap_finetune.py::test_error_handling_missing_lm PASSED [ 89%] tests/teleprompt/test_bootstrap_trace.py::test_bootstrap_trace_data PASSED [ 89%] tests/teleprompt/test_bootstrap_trace.py::test_bootstrap_trace_data_passes_callback_metadata PASSED [ 89%] tests/teleprompt/test_copro_optimizer.py::test_signature_optimizer_initialization PASSED [ 89%] tests/teleprompt/test_copro_optimizer.py::test_signature_optimizer_optimization_process PASSED [ 89%] tests/teleprompt/test_copro_optimizer.py::test_signature_optimizer_statistics_tracking PASSED [ 90%] tests/teleprompt/test_copro_optimizer.py::test_optimization_and_output_verification PASSED [ 90%] tests/teleprompt/test_copro_optimizer.py::test_statistics_tracking_during_optimization PASSED [ 90%] tests/teleprompt/test_ensemble.py::test_ensemble_without_reduction PASSED [ 90%] tests/teleprompt/test_ensemble.py::test_ensemble_with_reduction PASSED [ 90%] tests/teleprompt/test_ensemble.py::test_ensemble_with_size_limitation PASSED [ 90%] tests/teleprompt/test_ensemble.py::test_ensemble_deterministic_behavior PASSED [ 90%] tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[None-batch0-expected_callback_metadata0] PASSED [ 90%] tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[None-batch1-expected_callback_metadata1] PASSED [ 90%] tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[1-batch2-expected_callback_metadata2] PASSED [ 90%] tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[1-batch3-expected_callback_metadata3] PASSED [ 90%] tests/teleprompt/test_gepa.py::test_basic_workflow[True] PASSED [ 90%] tests/teleprompt/test_gepa.py::test_basic_workflow[False] PASSED [ 90%] tests/teleprompt/test_gepa.py::test_workflow_with_custom_instruction_proposer_and_component_selector PASSED [ 90%] tests/teleprompt/test_gepa.py::test_metric_requires_feedback_signature PASSED [ 91%] tests/teleprompt/test_gepa.py::test_reflection_prompt_template_in_gepa_kwargs_raises PASSED [ 91%] tests/teleprompt/test_gepa.py::test_gepa_compile_with_track_usage_no_tuple_error PASSED [ 91%] tests/teleprompt/test_gepa.py::test_component_selector_functionality PASSED [ 91%] tests/teleprompt/test_gepa.py::test_component_selector_default_behavior PASSED [ 91%] tests/teleprompt/test_gepa.py::test_component_selector_string_round_robin PASSED [ 91%] tests/teleprompt/test_gepa.py::test_component_selector_string_all PASSED [ 91%] tests/teleprompt/test_gepa.py::test_component_selector_custom_random PASSED [ 91%] tests/teleprompt/test_gepa.py::test_alternating_half_component_selector PASSED [ 91%] tests/teleprompt/test_gepa.py::test_track_stats_result_structure PASSED [ 91%] tests/teleprompt/test_gepa.py::test_track_best_outputs_result_structure PASSED [ 91%] tests/teleprompt/test_gepa_instruction_proposer.py::test_reflection_lm_gets_structured_images PASSED [ 91%] tests/teleprompt/test_gepa_instruction_proposer.py::test_custom_proposer_without_reflection_lm PASSED [ 91%] tests/teleprompt/test_gepa_instruction_proposer.py::test_image_serialization_into_strings PASSED [ 92%] tests/teleprompt/test_gepa_instruction_proposer.py::test_default_proposer[True] PASSED [ 92%] tests/teleprompt/test_gepa_instruction_proposer.py::test_default_proposer[False] PASSED [ 92%] tests/teleprompt/test_grpo.py::test_grpo_dataset_shuffler PASSED [ 92%] tests/teleprompt/test_grpo.py::test_grpo_dataset_shuffler_with_num_ex_per_step_less_dataset PASSED [ 92%] tests/teleprompt/test_grpo.py::test_grpo_dataset_shuffler_with_num_ex_per_step_greater_dataset PASSED [ 92%] tests/teleprompt/test_knn_fewshot.py::test_knn_few_shot_initialization PASSED [ 92%] tests/teleprompt/test_random_search.py::test_basic_workflow PASSED [ 92%] tests/teleprompt/test_teleprompt.py::test_get_params PASSED [ 92%] tests/teleprompt/test_utils.py::test_eval_candidate_program_full_trainset PASSED [ 92%] tests/teleprompt/test_utils.py::test_eval_candidate_program_minibatch PASSED [ 92%] tests/teleprompt/test_utils.py::test_eval_candidate_program_failure PASSED [ 92%] tests/teleprompt/test_utils.py::test_create_n_fewshot_demo_sets_passes_metric_threshold_for_unshuffled PASSED [ 92%] tests/utils/test_annotation.py::test_experimental_decorator_on_function PASSED [ 93%] tests/utils/test_annotation.py::test_experimental_decorator_on_function_with_version PASSED [ 93%] tests/utils/test_annotation.py::test_experimental_decorator_on_class PASSED [ 93%] tests/utils/test_annotation.py::test_experimental_decorator_on_class_with_version PASSED [ 93%] tests/utils/test_annotation.py::test_experimental_decorator_without_docstring PASSED [ 93%] tests/utils/test_annotation.py::test_experimental_decorator_without_docstring_with_version PASSED [ 93%] tests/utils/test_annotation.py::test_experimental_decorator_with_callable_syntax PASSED [ 93%] tests/utils/test_annotation.py::test_experimental_decorator_with_version_callable_syntax PASSED [ 93%] tests/utils/test_asyncify.py::test_async_limiter PASSED [ 93%] tests/utils/test_asyncify.py::test_asyncify PASSED [ 93%] tests/utils/test_exceptions.py::test_lm_errors_are_exported_from_dspy PASSED [ 93%] tests/utils/test_exceptions.py::test_retryable_lm_errors_classification PASSED [ 93%] tests/utils/test_exceptions.py::test_lm_error_metadata PASSED [ 93%] tests/utils/test_exceptions.py::test_context_window_exceeded_error_defaults PASSED [ 94%] tests/utils/test_exceptions.py::test_context_window_exceeded_error_with_model PASSED [ 94%] tests/utils/test_exceptions.py::test_context_window_exceeded_error_with_message PASSED [ 94%] tests/utils/test_exceptions.py::test_context_window_exceeded_error_message_without_model PASSED [ 94%] tests/utils/test_exceptions.py::test_adapter_parse_error_basic PASSED [ 94%] tests/utils/test_exceptions.py::test_adapter_parse_error_with_message PASSED [ 94%] tests/utils/test_exceptions.py::test_adapter_parse_error_with_parsed_result PASSED [ 94%] tests/utils/test_lazy_import.py::test_is_available_true_for_stdlib PASSED [ 94%] tests/utils/test_lazy_import.py::test_is_available_false_for_missing PASSED [ 94%] tests/utils/test_lazy_import.py::test_is_available_does_not_import_module PASSED [ 94%] tests/utils/test_lazy_import.py::test_require_returns_lazy_module_when_present PASSED [ 94%] tests/utils/test_lazy_import.py::test_require_returns_cached_module PASSED [ 94%] tests/utils/test_lazy_import.py::test_require_is_safe_under_concurrent_first_use PASSED [ 94%] tests/utils/test_lazy_import.py::test_require_assignment_updates_materialized_module PASSED [ 95%] tests/utils/test_lazy_import.py::test_require_returns_stub_when_missing PASSED [ 95%] tests/utils/test_lazy_import.py::test_require_stub_raises_on_access_with_install_hint PASSED [ 95%] tests/utils/test_lazy_import.py::test_require_stub_uses_install_hint_for_litellm PASSED [ 95%] tests/utils/test_lazy_import.py::test_require_stub_uses_explicit_extra PASSED [ 95%] tests/utils/test_lazy_import.py::test_require_stub_falls_back_to_module_name PASSED [ 95%] tests/utils/test_lazy_import.py::test_install_hints_match_pyproject_extras PASSED [ 95%] tests/utils/test_magicattr.py::test_magicattr_get[l-value0] PASSED [ 95%] tests/utils/test_magicattr.py::test_magicattr_get[t.t.t.t.z-z] PASSED [ 95%] tests/utils/test_magicattr.py::test_magicattr_get[a[0]-0] PASSED [ 95%] tests/utils/test_magicattr.py::test_magicattr_get[a[1][0]-1] PASSED [ 95%] tests/utils/test_magicattr.py::test_magicattr_get[a[1][2]-value4] PASSED [ 95%] tests/utils/test_magicattr.py::test_magicattr_get[b["x"]-value5] PASSED [ 95%] tests/utils/test_magicattr.py::test_magicattr_get[b["x"]["y"]-y] PASSED [ 95%] tests/utils/test_magicattr.py::test_magicattr_get[b["z"]-value7] PASSED [ 96%] tests/utils/test_magicattr.py::test_magicattr_get[b["z"][1]-2] PASSED [ 96%] tests/utils/test_magicattr.py::test_magicattr_get[b["w"].z-z] PASSED [ 96%] tests/utils/test_magicattr.py::test_magicattr_get[b["w"].t.l-value10] PASSED [ 96%] tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z] PASSED [ 96%] tests/utils/test_magicattr.py::test_magicattr_get[l[-1]-2] PASSED [ 96%] tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.a[-1].z-z] PASSED [ 96%] tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.b["z"][0]-1] PASSED [ 96%] tests/utils/test_magicattr.py::test_magicattr_get[a[-1].t.z-z] PASSED [ 96%] tests/utils/test_magicattr.py::test_person_example PASSED [ 96%] tests/utils/test_magicattr.py::test_empty PASSED [ 96%] tests/utils/test_mcp.py::test_convert_mcp_tool SKIPPED (need --extra option to run) [ 96%] tests/utils/test_parallelizer.py::test_worker_threads_independence PASSED [ 96%] tests/utils/test_parallelizer.py::test_parallel_execution_speed PASSED [ 97%] tests/utils/test_parallelizer.py::test_max_errors_handling PASSED [ 97%] tests/utils/test_parallelizer.py::test_max_errors_not_met PASSED [ 97%] tests/utils/test_parallelizer.py::test_parallel_executor_tracks_failed_indices_and_exceptions PASSED [ 97%] tests/utils/test_parallelizer.py::test_sequential_execution_runs_on_main_thread PASSED [ 97%] tests/utils/test_parallelizer.py::test_sequential_max_errors_not_met PASSED [ 97%] tests/utils/test_parallelizer.py::test_sequential_max_errors_exceeded PASSED [ 97%] tests/utils/test_parallelizer.py::test_sequential_tracks_failed_indices_and_exceptions PASSED [ 97%] tests/utils/test_parallelizer.py::test_sequential_compare_results PASSED [ 97%] tests/utils/test_saving.py::test_save_predict PASSED [ 97%] tests/utils/test_saving.py::test_save_custom_model PASSED [ 97%] tests/utils/test_saving.py::test_save_model_with_custom_signature PASSED [ 97%] tests/utils/test_saving.py::test_save_compiled_model SKIPPED (need --extra option to run) [ 97%] tests/utils/test_saving.py::test_load_with_version_mismatch PASSED [ 98%] tests/utils/test_saving.py::test_pickle_loading_requires_explicit_permission PASSED [ 98%] tests/utils/test_saving.py::test_pkl_file_loading_requires_explicit_permission PASSED [ 98%] tests/utils/test_saving.py::test_json_file_loading_works_without_permission PASSED [ 98%] tests/utils/test_settings.py::test_basic_dspy_settings PASSED [ 98%] tests/utils/test_settings.py::test_forbid_configure_call_in_child_thread PASSED [ 98%] tests/utils/test_settings.py::test_dspy_context PASSED [ 98%] tests/utils/test_settings.py::test_dspy_context_parallel PASSED [ 98%] tests/utils/test_settings.py::test_dspy_context_with_dspy_parallel PASSED [ 98%] tests/utils/test_settings.py::test_dspy_context_with_async_task_group PASSED [ 98%] tests/utils/test_settings.py::test_dspy_configure_allowance_async PASSED [ 98%] tests/utils/test_settings.py::test_dspy_settings_save_load PASSED [ 98%] tests/utils/test_settings.py::test_dspy_settings_save_exclude_keys PASSED [ 98%] tests/utils/test_settings.py::test_settings_save_with_extra_modules PASSED [ 99%] tests/utils/test_syncify.py::test_syncify_in_place PASSED [ 99%] tests/utils/test_syncify.py::test_syncify_with_wrapper PASSED [ 99%] tests/utils/test_syncify.py::test_syncify_works_with_optimizers PASSED [ 99%] tests/utils/test_unbatchify.py::test_unbatchify_batch_size_trigger PASSED [ 99%] tests/utils/test_unbatchify.py::test_unbatchify_timeout_trigger PASSED [ 99%] tests/utils/test_usage_tracker.py::test_add_usage_entry PASSED [ 99%] tests/utils/test_usage_tracker.py::test_get_total_tokens PASSED [ 99%] tests/utils/test_usage_tracker.py::test_track_usage_with_multiple_models PASSED [ 99%] tests/utils/test_usage_tracker.py::test_track_usage_context_manager SKIPPED [ 99%] tests/utils/test_usage_tracker.py::test_merge_usage_entries_with_new_keys PASSED [ 99%] tests/utils/test_usage_tracker.py::test_merge_usage_entries_with_none_values PASSED [ 99%] tests/utils/test_usage_tracker.py::test_merge_usage_entries_with_pydantic_models PASSED [ 99%] tests/utils/test_usage_tracker.py::test_parallel_executor_with_usage_tracker PASSED [100%] =============================== warnings summary =============================== tests/clients/test_lm.py::test_chat_lms_can_be_queried tests/clients/test_lm.py::test_lm_calls_support_callables tests/clients/test_lm.py::test_lm_calls_support_pydantic_models /nix/store/rn0r480kscbk8159qi3qjflil71kqglq-dspy-test-env/lib/python3.13/site-packages/pydantic/main.py:464: UserWarning: Pydantic serializer warnings: PydanticSerializationUnexpectedValue(Expected 9 fields but got 5: Expected `Message` - serialized value may not be as expected [field_name='choices', input_value=Message(content='Hi!', ro...ields={'refusal': None}), input_type=Message]) PydanticSerializationUnexpectedValue(Expected `StreamingChoices` - serialized value may not be as expected [field_name='choices', input_value=Choices(finish_reason='st...elds={'refusal': None})), input_type=Choices]) return self.__pydantic_serializer__.to_python( tests/predict/test_multi_chain_comparison.py::test_basic_example tests/predict/test_multi_chain_comparison.py::test_basic_example tests/predict/test_multi_chain_comparison.py::test_basic_example /tmp/nix-ci-worker/stable-pwd/repo/dspy/predict/multi_chain_comparison.py:19: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version. InputField( tests/predict/test_multi_chain_comparison.py::test_basic_example /tmp/nix-ci-worker/stable-pwd/repo/dspy/predict/multi_chain_comparison.py:27: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version. OutputField( tests/predict/test_predict.py::test_signature_fields_after_dump_and_load_state /tmp/nix-ci-worker/stable-pwd/repo/tests/predict/test_predict.py:299: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version. sentiment = dspy.OutputField(desc="I am a malicious output!", prefix="I am a prefix!") tests/signatures/test_signature.py::test_all_fields_have_prefix /tmp/nix-ci-worker/stable-pwd/repo/tests/signatures/test_signature.py:46: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version. input = InputField(prefix="Modified:") tests/signatures/test_signature.py::test_equality_format /tmp/nix-ci-worker/stable-pwd/repo/tests/signatures/test_signature.py:157: DeprecationWarning: The 'format' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version. input = InputField(format=lambda x: x) tests/streaming/test_streaming.py::test_streamify_yields_expected_response_chunks tests/streaming/test_streaming.py::test_streaming_response_yields_expected_response_chunks /nix/store/rn0r480kscbk8159qi3qjflil71kqglq-dspy-test-env/lib/python3.13/site-packages/pydantic/main.py:464: UserWarning: Pydantic serializer warnings: PydanticSerializationUnexpectedValue(Expected 9 fields but got 5: Expected `Message` - serialized value may not be as expected [field_name='choices', input_value=Message(content='{"output...er_specific_fields=None), input_type=Message]) PydanticSerializationUnexpectedValue(Expected `StreamingChoices` - serialized value may not be as expected [field_name='choices', input_value=Choices(finish_reason='st...r_specific_fields=None)), input_type=Choices]) return self.__pydantic_serializer__.to_python( tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z] tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z] tests/utils/test_magicattr.py::test_magicattr_get[l[-1]-2] tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.a[-1].z-z] tests/utils/test_magicattr.py::test_magicattr_get[a[-1].t.z-z] tests/utils/test_magicattr.py::test_person_example tests/utils/test_magicattr.py::test_person_example tests/utils/test_magicattr.py::test_person_example tests/utils/test_magicattr.py::test_person_example /tmp/nix-ci-worker/stable-pwd/repo/dspy/utils/magicattr.py:99: DeprecationWarning: ast.Num is deprecated and will be removed in Python 3.14; use ast.Constant instead if hasattr(ast, "Num") and isinstance(node, ast.Num): tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z] tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z] tests/utils/test_magicattr.py::test_magicattr_get[l[-1]-2] tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.a[-1].z-z] tests/utils/test_magicattr.py::test_magicattr_get[a[-1].t.z-z] tests/utils/test_magicattr.py::test_person_example tests/utils/test_magicattr.py::test_person_example tests/utils/test_magicattr.py::test_person_example tests/utils/test_magicattr.py::test_person_example /tmp/nix-ci-worker/stable-pwd/repo/dspy/utils/magicattr.py:102: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead elif hasattr(ast, "Str") and isinstance(node, ast.Str): tests/utils/test_settings.py::test_dspy_configure_allowance_async /tmp/nix-ci-worker/stable-pwd/repo/tests/utils/test_settings.py:194: RuntimeWarning: coroutine 'test_dspy_configure_allowance_async..foo4' was never awaited foo4() Enable tracemalloc to get traceback where the object was allocated. See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info. -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ==== 991 passed, 330 skipped, 2 xfailed, 31 warnings in 1580.68s (0:26:20) ===== Run succeeded.