$ /nix/store/9ypz3flqsrl5xl495mm8h645gadjsxi1-coreutils-9.11/bin/timeout --kill-after=15s 7200s /nix/store/23zk8sgzamrbmj1fday15szf7f2279s8-nix-2.34.7/bin/nix --extra-experimental-features nix-command --extra-experimental-features flakes --log-format internal-json build --no-link --print-out-paths git+https://github.com/NorfairKing/dspy?ref=your-first-flake&rev=453ecf257a343742170b3e1cef20271508c3e6a4#checks.x86_64-linux.pytest --print-build-logs

Building /nix/store/rcq49d990xqb45ygc83sng306xk2cspr-dspy-pytest.drv
[dspy-pytest] Phase: unpackPhase
[dspy-pytest] unpacking source archive /nix/store/65n02b36yl4dm50hw7nc4inq18zbw5rm-r9a1svd8gqmcs0gcy8lz2ssr6798r95l-source
[dspy-pytest] source root is r9a1svd8gqmcs0gcy8lz2ssr6798r95l-source
[dspy-pytest] Phase: patchPhase
[dspy-pytest] Phase: updateAutotoolsGnuConfigScriptsPhase
[dspy-pytest] Phase: checkPhase
[dspy-pytest] ============================= test session starts ==============================
[dspy-pytest] platform linux -- Python 3.13.13, pytest-8.4.2, pluggy-1.6.0 -- /nix/store/rn0r480kscbk8159qi3qjflil71kqglq-dspy-test-env/bin/python3.13
[dspy-pytest] rootdir: /build/r9a1svd8gqmcs0gcy8lz2ssr6798r95l-source
[dspy-pytest] configfile: pyproject.toml
[dspy-pytest] plugins: anyio-4.13.0, asyncio-1.3.0, typeguard-4.5.1, mock-3.14.1
[dspy-pytest] asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
[dspy-pytest] collected 1322 items / 9 deselected / 1 skipped / 1313 selected                
[dspy-pytest] 
[dspy-pytest] tests/adapters/test_adapter_utils.py::test_parse_value_str_annotation PASSED [  0%]
[dspy-pytest] tests/adapters/test_adapter_utils.py::test_parse_value_pydantic_types PASSED [  0%]
[dspy-pytest] tests/adapters/test_adapter_utils.py::test_parse_value_basic_types PASSED [  0%]
[dspy-pytest] tests/adapters/test_adapter_utils.py::test_parse_value_literal PASSED    [  0%]
[dspy-pytest] tests/adapters/test_adapter_utils.py::test_parse_value_union PASSED      [  0%]
[dspy-pytest] tests/adapters/test_adapter_utils.py::test_parse_value_json_repair PASSED [  0%]
[dspy-pytest] tests/adapters/test_audio.py::test_normalize_audio_format[wav-wav] PASSED [  0%]
[dspy-pytest] tests/adapters/test_audio.py::test_normalize_audio_format[mp3-mp3] PASSED [  0%]
[dspy-pytest] tests/adapters/test_audio.py::test_normalize_audio_format[x-wav-wav] PASSED [  0%]
[dspy-pytest] tests/adapters/test_audio.py::test_normalize_audio_format[x-mp3-mp3] PASSED [  0%]
[dspy-pytest] tests/adapters/test_audio.py::test_normalize_audio_format[x-flac-flac] PASSED [  0%]
[dspy-pytest] tests/adapters/test_audio.py::test_normalize_audio_format[my-x-format-my-x-format] PASSED [  0%]
[dspy-pytest] tests/adapters/test_audio.py::test_normalize_audio_format[x-my-format-my-format] PASSED [  0%]
[dspy-pytest] tests/adapters/test_audio.py::test_normalize_audio_format[-] PASSED      [  1%]
[dspy-pytest] tests/adapters/test_audio.py::test_normalize_audio_format[x--] PASSED    [  1%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_format_exact_messages_for_simple_signature_with_demo PASSED [  1%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_format_exact_messages_with_nested_output PASSED [  1%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_basic_schema_generation PASSED [  1%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_optional_fields PASSED [  1%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_primitive_types PASSED [  1%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_lists_with_bracket_notation PASSED [  1%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_complex_nested_models PASSED [  1%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_raise_error_on_circular_references PASSED [  1%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_formats_pydantic_inputs_as_clean_json PASSED [  1%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_mixed_input_types PASSED [  1%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_schema_generation_errors_gracefully PASSED [  1%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_raises_on_missing_fields PASSED [  2%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_handles_type_casting_errors PASSED [  2%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_with_images PASSED [  2%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_with_tools PASSED [  2%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_with_code PASSED  [  2%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_with_conversation_history PASSED [  2%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_vs_json_adapter_token_efficiency PASSED [  2%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_vs_json_adapter_functional_compatibility PASSED [  2%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_async_functionality PASSED [  2%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_with_field_aliases PASSED [  2%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_field_alias_without_description PASSED [  2%]
[dspy-pytest] tests/adapters/test_baml_adapter.py::test_baml_adapter_multiple_pydantic_input_fields PASSED [  2%]
[dspy-pytest] tests/adapters/test_base_type.py::test_basic_extract_custom_type_from_annotation PASSED [  2%]
[dspy-pytest] tests/adapters/test_base_type.py::test_extract_custom_type_from_annotation_with_nested_type PASSED [  3%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-two-Literal['one', 'two', 'three"']-Literal['four', 'five', 'six"']] PASSED [  3%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-she's here-Literal["she's here", 'okay', 'test']-Literal['done', "maybe'soon", 'later']] PASSED [  3%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-another-Literal['both"and\\'', 'another']-Literal['yet"another\\'', 'plain']] PASSED [  3%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-foo-Literal['foo', 'bar']-Literal['baz', 'qux']] PASSED [  3%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_quotes_literals_as_expected[Literal-Literal-bar-Literal[1, 'bar']-Literal[True, 3, 'foo']] PASSED [  3%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_sync_call PASSED  [  3%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_async_call PASSED [  3%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_for_simple_signature PASSED [  3%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_demo_and_typed_outputs PASSED [  3%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_nested_pydantic_models PASSED [  3%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_incomplete_demo PASSED [  3%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_history PASSED [  3%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_list_value_for_string_input PASSED [  4%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_literal_output PASSED [  4%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_multimodal_custom_type_inputs PASSED [  4%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_history_demo_pydantic_tools_and_image PASSED [  4%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_base_custom_type_input PASSED [  4%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_citations_output_demo PASSED [  4%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_and_lm_kwargs_with_native_citations PASSED [  4%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_preserves_passthrough_lm_kwargs PASSED [  4%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_and_lm_kwargs_with_native_reasoning PASSED [  4%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_calling_still_enables_native_reasoning PASSED [  4%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_nonnative_strips_native_tool_kwargs PASSED [  4%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_reasoning_and_code_outputs PASSED [  4%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_and_lm_kwargs_with_native_tool_calling PASSED [  4%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_can_request_parallel_tool_calls[adapter0] PASSED [  5%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_can_request_parallel_tool_calls[adapter1] PASSED [  5%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_can_request_parallel_tool_calls[adapter2] PASSED [  5%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_adapter_native_tool_calling_respects_lm_kwargs_parallel_tool_call_override PASSED [  5%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_replay PASSED [  5%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_replays_parallel_tool_results PASSED [  5%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_empty_user_message PASSED [  5%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_unmatched_tool_calls[call_1-None] PASSED [  5%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_unmatched_tool_calls[call_1-tool_call_results1] PASSED [  5%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_tool_history_skips_unmatched_tool_calls[None-tool_call_results2] PASSED [  5%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_non_native_tool_history PASSED [  5%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_non_native_tool_history_remains_text_based[adapter0] PASSED [  5%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_non_native_tool_history_remains_text_based[adapter1] PASSED [  5%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_accepts_custom_history_formatter_returning_messages_only PASSED [  6%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_with_tool_input PASSED [  6%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_format_exact_messages_kitchen_sink PASSED [  6%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_with_pydantic_models PASSED [  6%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_signature_information PASSED [  6%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_exception_raised_on_failure PASSED [  6%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image PASSED [  6%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image_with_few_shot_examples PASSED [  6%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image_with_nested_images PASSED [  6%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_image_with_few_shot_examples_with_nested_images PASSED [  6%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_with_tool PASSED  [  6%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_with_code PASSED  [  6%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_code_output_field_omits_json_schema_in_prompt PASSED [  6%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_citations_output_field_keeps_json_schema_in_prompt PASSED [  7%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_formats_conversation_history PASSED [  7%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_fallback_to_json_adapter_on_exception PASSED [  7%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_fallback_preserves_native_function_calling_flag PASSED [  7%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_respects_use_json_adapter_fallback_flag PASSED [  7%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_fallback_to_json_adapter_on_exception_async PASSED [  7%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_async_fallback_preserves_native_function_calling_flag PASSED [  7%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_toolcalls_native_function_calling PASSED [  7%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_toolcalls_vague_match PASSED [  7%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_native_reasoning PASSED [  7%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_chat_adapter_parses_float_with_underscores PASSED [  7%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_format_system_message PASSED   [  7%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_null_content_raises_adapter_parse_error PASSED [  7%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_empty_string_content_raises_adapter_parse_error PASSED [  7%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_tool_call_with_null_content_does_not_raise PASSED [  8%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_tool_call_with_unstructured_content_does_not_raise PASSED [  8%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_tool_call_with_structured_content_preserves_other_outputs PASSED [  8%]
[dspy-pytest] tests/adapters/test_chat_adapter.py::test_provider_tool_calls_preserve_id_and_repair_arguments PASSED [  8%]
[dspy-pytest] tests/adapters/test_citation.py::test_citation_validate_input PASSED     [  8%]
[dspy-pytest] tests/adapters/test_citation.py::test_citations_in_nested_type PASSED    [  8%]
[dspy-pytest] tests/adapters/test_citation.py::test_citation_with_all_fields PASSED    [  8%]
[dspy-pytest] tests/adapters/test_citation.py::test_citation_format PASSED             [  8%]
[dspy-pytest] tests/adapters/test_citation.py::test_citations_format PASSED            [  8%]
[dspy-pytest] tests/adapters/test_citation.py::test_citations_from_dict_list PASSED    [  8%]
[dspy-pytest] tests/adapters/test_citation.py::test_citations_postprocessing PASSED    [  8%]
[dspy-pytest] tests/adapters/test_citation.py::test_citation_extraction_from_lm_response PASSED [  8%]
[dspy-pytest] tests/adapters/test_code.py::test_code_validate_input PASSED             [  8%]
[dspy-pytest] tests/adapters/test_code.py::test_code_in_nested_type PASSED             [  9%]
[dspy-pytest] tests/adapters/test_code.py::test_code_with_language PASSED              [  9%]
[dspy-pytest] tests/adapters/test_code.py::test_code_parses_from_dirty_code PASSED     [  9%]
[dspy-pytest] tests/adapters/test_document.py::test_document_validate_input PASSED     [  9%]
[dspy-pytest] tests/adapters/test_document.py::test_document_in_nested_type PASSED     [  9%]
[dspy-pytest] tests/adapters/test_document.py::test_document_with_all_fields PASSED    [  9%]
[dspy-pytest] tests/adapters/test_document.py::test_document_format PASSED             [  9%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_for_simple_signature PASSED [  9%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_demo_and_typed_output PASSED [  9%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_described_and_bool_outputs PASSED [  9%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_history_demo_pydantic_tools_and_image PASSED [  9%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_int_and_mapping_outputs PASSED [  9%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_literal_and_enum_outputs PASSED [  9%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_nested_pydantic_output PASSED [ 10%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_incomplete_demo PASSED [ 10%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_and_lm_kwargs_with_native_tool_calling PASSED [ 10%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_messages_with_tool_calls_output_demo PASSED [ 10%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_format_exact_non_native_tool_result_history_field PASSED [ 10%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_passes_structured_output_when_supported_by_model PASSED [ 10%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_not_using_structured_outputs_when_not_supported_by_model PASSED [ 10%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_with_structured_outputs_does_not_mutate_original_signature PASSED [ 10%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_sync_call PASSED  [ 10%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_async_call PASSED [ 10%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_on_pydantic_model PASSED [ 10%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_parse_raise_error_on_mismatch_fields PASSED [ 10%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_formats_image PASSED [ 10%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_formats_image_with_few_shot_examples PASSED [ 11%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_formats_image_with_nested_images PASSED [ 11%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_formats_with_nested_documents PASSED [ 11%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_formats_image_with_few_shot_examples_with_nested_images PASSED [ 11%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_with_tool PASSED  [ 11%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_with_code PASSED  [ 11%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_formats_conversation_history PASSED [ 11%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_on_pydantic_model_async PASSED [ 11%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_does_not_fallback_to_json_mode_on_structured_output_lm_error PASSED [ 11%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_json_mode_no_structured_outputs PASSED [ 11%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_json_mode_no_structured_outputs_async PASSED [ 11%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_does_not_fallback_to_json_mode_on_structured_output_lm_error_async PASSED [ 11%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_error_message_on_json_adapter_failure PASSED [ 11%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_error_message_on_json_adapter_failure_async PASSED [ 12%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_toolcalls_native_function_calling PASSED [ 12%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_toolcalls_no_native_function_calling PASSED [ 12%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_native_reasoning PASSED [ 12%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_json_adapter_with_responses_api PASSED [ 12%]
[dspy-pytest] tests/adapters/test_json_adapter.py::test_format_system_message PASSED   [ 12%]
[dspy-pytest] tests/adapters/test_reasoning.py::test_reasoning_basic_operations PASSED [ 12%]
[dspy-pytest] tests/adapters/test_reasoning.py::test_reasoning_concatenation PASSED    [ 12%]
[dspy-pytest] tests/adapters/test_reasoning.py::test_reasoning_string_methods PASSED   [ 12%]
[dspy-pytest] tests/adapters/test_reasoning.py::test_reasoning_with_chain_of_thought PASSED [ 12%]
[dspy-pytest] tests/adapters/test_reasoning.py::test_reasoning_error_message PASSED    [ 12%]
[dspy-pytest] tests/adapters/test_tool.py::test_basic_initialization PASSED            [ 12%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_from_function PASSED              [ 12%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_from_class PASSED                 [ 13%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_from_function_with_pydantic PASSED [ 13%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_from_function_with_pydantic_nesting PASSED [ 13%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_callable PASSED                   [ 13%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_with_pydantic_callable PASSED     [ 13%]
[dspy-pytest] tests/adapters/test_tool.py::test_invalid_function_call PASSED           [ 13%]
[dspy-pytest] tests/adapters/test_tool.py::test_parameter_desc PASSED                  [ 13%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_with_default_args_without_type_hints PASSED [ 13%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_call_parses_args PASSED           [ 13%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_call_parses_nested_list_of_pydantic_model PASSED [ 13%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_call_kwarg PASSED                 [ 13%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_str PASSED                        [ 13%]
[dspy-pytest] tests/adapters/test_tool.py::test_async_tool_from_function PASSED        [ 13%]
[dspy-pytest] tests/adapters/test_tool.py::test_async_tool_with_pydantic PASSED        [ 14%]
[dspy-pytest] tests/adapters/test_tool.py::test_async_tool_with_complex_pydantic PASSED [ 14%]
[dspy-pytest] tests/adapters/test_tool.py::test_async_tool_invalid_call PASSED         [ 14%]
[dspy-pytest] tests/adapters/test_tool.py::test_async_tool_with_kwargs PASSED          [ 14%]
[dspy-pytest] tests/adapters/test_tool.py::test_async_concurrent_calls PASSED          [ 14%]
[dspy-pytest] tests/adapters/test_tool.py::test_async_tool_call_in_sync_mode PASSED    [ 14%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data0-expected0] PASSED [ 14%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data1-expected1] PASSED [ 14%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data2-expected2] PASSED [ 14%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_calls_format_basic[tool_calls_data3-expected3] PASSED [ 14%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_calls_format_from_dict_list PASSED [ 14%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_calls_preserves_ids_from_dict_list_and_format_omits_ids PASSED [ 14%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_calls_json_schema_omits_internal_id_field PASSED [ 14%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_calls_can_carry_results_without_formatting_them_for_lm PASSED [ 15%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_call_results_from_tool_calls_and_values PASSED [ 15%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_call_results_history_serialization_round_trip PASSED [ 15%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_call_results_can_round_trip_as_native_tool_result_message PASSED [ 15%]
[dspy-pytest] tests/adapters/test_tool.py::test_toolcalls_vague_match PASSED           [ 15%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_convert_input_schema_to_tool_args_no_input_params PASSED [ 15%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_convert_input_schema_to_tool_args_lang_chain PASSED [ 15%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_call_execute PASSED               [ 15%]
[dspy-pytest] tests/adapters/test_tool.py::test_tool_call_execute_with_local_functions PASSED [ 15%]
[dspy-pytest] tests/adapters/test_two_step_adapter.py::test_two_step_adapter_format_exact_messages_for_simple_signature_with_demo PASSED [ 15%]
[dspy-pytest] tests/adapters/test_two_step_adapter.py::test_two_step_adapter_format_exact_messages_with_typed_outputs PASSED [ 15%]
[dspy-pytest] tests/adapters/test_two_step_adapter.py::test_two_step_adapter_call PASSED [ 15%]
[dspy-pytest] tests/adapters/test_two_step_adapter.py::test_two_step_adapter_async_call PASSED [ 15%]
[dspy-pytest] tests/adapters/test_two_step_adapter.py::test_two_step_adapter_parse PASSED [ 15%]
[dspy-pytest] tests/adapters/test_two_step_adapter.py::test_two_step_adapter_parse_errors PASSED [ 16%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_and_parse_basic PASSED [ 16%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_multiple_fields PASSED [ 16%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_raises_on_missing_field PASSED [ 16%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_casts_types PASSED [ 16%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_parse_raises_on_type_error PASSED [ 16%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_and_parse_nested_model PASSED [ 16%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_and_parse_list_of_models PASSED [ 16%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_with_tool_like_output PASSED [ 16%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_formats_nested_images PASSED [ 16%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_with_code PASSED    [ 16%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_full_prompt PASSED  [ 16%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_for_simple_signature PASSED [ 16%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_non_native_tool_result_history_field PASSED [ 17%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_for_two_input_signature PASSED [ 17%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_demo_and_typed_output PASSED [ 17%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_history_demo_pydantic_tools_and_image PASSED [ 17%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_nested_pydantic_output PASSED [ 17%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_xml_adapter_format_exact_messages_with_incomplete_demo PASSED [ 17%]
[dspy-pytest] tests/adapters/test_xml_adapter.py::test_format_system_message PASSED    [ 17%]
[dspy-pytest] tests/callback/test_callback.py::test_callback_injection[args0-kwargs0] PASSED [ 17%]
[dspy-pytest] tests/callback/test_callback.py::test_callback_injection[args1-kwargs1] PASSED [ 17%]
[dspy-pytest] tests/callback/test_callback.py::test_callback_injection[args2-kwargs2] PASSED [ 17%]
[dspy-pytest] tests/callback/test_callback.py::test_callback_injection[args3-kwargs3] PASSED [ 17%]
[dspy-pytest] tests/callback/test_callback.py::test_callback_injection_local PASSED    [ 17%]
[dspy-pytest] tests/callback/test_callback.py::test_callback_error_handling PASSED     [ 17%]
[dspy-pytest] tests/callback/test_callback.py::test_multiple_callbacks PASSED          [ 18%]
[dspy-pytest] tests/callback/test_callback.py::test_callback_complex_module PASSED     [ 18%]
[dspy-pytest] tests/callback/test_callback.py::test_callback_async_module PASSED       [ 18%]
[dspy-pytest] tests/callback/test_callback.py::test_tool_calls PASSED                  [ 18%]
[dspy-pytest] tests/callback/test_callback.py::test_active_id PASSED                   [ 18%]
[dspy-pytest] tests/clients/test_cache.py::test_initialization PASSED                  [ 18%]
[dspy-pytest] tests/clients/test_cache.py::test_invalid_cache_initialization PASSED    [ 18%]
[dspy-pytest] tests/clients/test_cache.py::test_cache_key_generation PASSED            [ 18%]
[dspy-pytest] tests/clients/test_cache.py::test_put_and_get PASSED                     [ 18%]
[dspy-pytest] tests/clients/test_cache.py::test_cache_miss PASSED                      [ 18%]
[dspy-pytest] tests/clients/test_cache.py::test_unserializable_key PASSED              [ 18%]
[dspy-pytest] tests/clients/test_cache.py::test_reset_memory_cache PASSED              [ 18%]
[dspy-pytest] tests/clients/test_cache.py::test_save_and_load_memory_cache PASSED      [ 18%]
[dspy-pytest] tests/clients/test_cache.py::test_request_cache_decorator PASSED         [ 19%]
[dspy-pytest] tests/clients/test_cache.py::test_request_cache_decorator_with_ignored_args_for_cache_key PASSED [ 19%]
[dspy-pytest] tests/clients/test_cache.py::test_request_cache_decorator_async PASSED   [ 19%]
[dspy-pytest] tests/clients/test_cache.py::test_cache_consistency_with_lm_call_modifies_the_request PASSED [ 19%]
[dspy-pytest] tests/clients/test_cache.py::test_cache_fallback_on_restricted_environment PASSED [ 19%]
[dspy-pytest] tests/clients/test_cache.py::test_cache_init_with_disk_disabled_and_none_dir PASSED [ 19%]
[dspy-pytest] tests/clients/test_cache.py::test_model_response_roundtrip_in_restricted_mode PASSED [ 19%]
[dspy-pytest] tests/clients/test_cache.py::test_registered_dataclass_roundtrip_in_restricted_mode PASSED [ 19%]
[dspy-pytest] tests/clients/test_cache.py::test_configure_cache_registers_safe_types PASSED [ 19%]
[dspy-pytest] tests/clients/test_cache.py::test_corrupt_disk_entries_return_none PASSED [ 19%]
[dspy-pytest] tests/clients/test_cache.py::test_restricted_and_unrestricted_share_wire_format PASSED [ 19%]
[dspy-pytest] tests/clients/test_cache.py::test_unlisted_type_rejected_on_read PASSED  [ 19%]
[dspy-pytest] tests/clients/test_cache.py::test_safe_types_rejects_non_types PASSED    [ 19%]
[dspy-pytest] tests/clients/test_databricks.py::test_create_directory_in_databricks_unity_catalog SKIPPED [ 20%]
[dspy-pytest] tests/clients/test_databricks.py::test_create_finetuning_job SKIPPED     [ 20%]
[dspy-pytest] tests/clients/test_databricks.py::test_deploy_finetuned_model SKIPPED    [ 20%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[dict] PASSED [ 20%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[list] PASSED [ 20%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[str] PASSED [ 20%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[int] PASSED [ 20%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_plain_values_roundtrip[tuple] PASSED [ 20%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_pydantic_roundtrip PASSED [ 20%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_dataclass_roundtrip PASSED [ 20%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_ndarray_roundtrip PASSED  [ 20%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_litellm_response_roundtrip PASSED [ 20%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_text_completion_response_roundtrip PASSED [ 20%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_embedding_response_roundtrip PASSED [ 21%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_responses_api_roundtrip PASSED [ 21%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_tool_call_response_roundtrip PASSED [ 21%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_cached_tool_call_normalizes_after_fresh_process_roundtrip PASSED [ 21%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_cached_responses_function_call_normalizes_after_fresh_process_roundtrip PASSED [ 21%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_unlisted_type_blocked_on_read PASSED [ 21%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_allowlists_are_isolated PASSED [ 21%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_numpy_ctypeslib_blocked PASSED [ 21%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_restricted_load_rejects_unknown_type PASSED [ 21%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_corrupt_pickle_raises_deserialization_error PASSED [ 21%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_nested_class_safe_types PASSED [ 21%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_restricted_load_allows_builtin_types PASSED [ 21%]
[dspy-pytest] tests/clients/test_disk_serialization.py::test_all_cached_lm_types_roundtrip PASSED [ 21%]
[dspy-pytest] tests/clients/test_embedding.py::test_litellm_embedding PASSED           [ 22%]
[dspy-pytest] tests/clients/test_embedding.py::test_callable_embedding PASSED          [ 22%]
[dspy-pytest] tests/clients/test_embedding.py::test_callable_numpy_embedding_persists_to_disk PASSED [ 22%]
[dspy-pytest] tests/clients/test_embedding.py::test_invalid_model_type PASSED          [ 22%]
[dspy-pytest] tests/clients/test_embedding.py::test_async_embedding PASSED             [ 22%]
[dspy-pytest] tests/clients/test_embedding.py::test_call_caching_false_overrides_instance_true PASSED [ 22%]
[dspy-pytest] tests/clients/test_embedding.py::test_call_caching_true_overrides_instance_false PASSED [ 22%]
[dspy-pytest] tests/clients/test_embedding.py::test_acall_caching_false_overrides_instance_true PASSED [ 22%]
[dspy-pytest] tests/clients/test_embedding.py::test_acall_caching_true_overrides_instance_false PASSED [ 22%]
[dspy-pytest] tests/clients/test_inspect_global_history.py::test_inspect_history_basic PASSED [ 22%]
[dspy-pytest] tests/clients/test_inspect_global_history.py::test_inspect_history_renders_message_tool_calls PASSED [ 22%]
[dspy-pytest] tests/clients/test_inspect_global_history.py::test_inspect_history_renders_output_tool_calls_without_text PASSED [ 22%]
[dspy-pytest] tests/clients/test_inspect_global_history.py::test_inspect_history_with_n PASSED [ 22%]
[dspy-pytest] tests/clients/test_inspect_global_history.py::test_inspect_empty_history PASSED [ 23%]
[dspy-pytest] tests/clients/test_inspect_global_history.py::test_inspect_history_n_larger_than_history PASSED [ 23%]
[dspy-pytest] tests/clients/test_lazy_litellm_import.py::test_import_dspy_does_not_import_litellm PASSED [ 23%]
[dspy-pytest] tests/clients/test_lazy_litellm_import.py::test_lm_litellm_use_raises_helpful_error_without_litellm PASSED [ 23%]
[dspy-pytest] tests/clients/test_lazy_litellm_import.py::test_embedder_litellm_use_raises_helpful_error_without_litellm PASSED [ 23%]
[dspy-pytest] tests/clients/test_lazy_litellm_import.py::test_concurrent_lm_first_use_materializes_litellm_once PASSED [ 23%]
[dspy-pytest] tests/clients/test_lm.py::test_chat_lms_can_be_queried PASSED            [ 23%]
[dspy-pytest] tests/clients/test_lm.py::test_dspy_cache PASSED                         [ 23%]
[dspy-pytest] tests/clients/test_lm.py::test_disabled_cache_skips_cache_key PASSED     [ 23%]
[dspy-pytest] tests/clients/test_lm.py::test_rollout_id_bypasses_cache PASSED          [ 23%]
[dspy-pytest] tests/clients/test_lm.py::test_zero_temperature_rollout_warns_once PASSED [ 23%]
[dspy-pytest] tests/clients/test_lm.py::test_rollout_id_with_default_temperature_does_not_warn PASSED [ 23%]
[dspy-pytest] tests/clients/test_lm.py::test_text_lms_can_be_queried PASSED            [ 23%]
[dspy-pytest] tests/clients/test_lm.py::test_lm_calls_support_callables PASSED         [ 23%]
[dspy-pytest] tests/clients/test_lm.py::test_lm_calls_support_pydantic_models PASSED   [ 24%]
[dspy-pytest] tests/clients/test_lm.py::test_lm_wraps_litellm_errors_with_metadata PASSED [ 24%]
[dspy-pytest] tests/clients/test_lm.py::test_lm_wraps_litellm_context_window_error PASSED [ 24%]
[dspy-pytest] tests/clients/test_lm.py::test_lm_wraps_unknown_boundary_error_as_unexpected_error PASSED [ 24%]
[dspy-pytest] tests/clients/test_lm.py::test_lm_preserves_existing_lm_error_without_self_cause PASSED [ 24%]
[dspy-pytest] tests/clients/test_lm.py::test_lm_preserves_existing_lm_error_without_self_cause_async PASSED [ 24%]
[dspy-pytest] tests/clients/test_lm.py::test_retry_number_set_correctly PASSED         [ 24%]
[dspy-pytest] tests/clients/test_lm.py::test_retry_made_on_system_errors PASSED        [ 24%]
[dspy-pytest] tests/clients/test_lm.py::test_reasoning_model_token_parameter PASSED    [ 24%]
[dspy-pytest] tests/clients/test_lm.py::test_reasoning_model_requirements[openai/o1] PASSED [ 24%]
[dspy-pytest] tests/clients/test_lm.py::test_reasoning_model_requirements[openai/gpt-5-nano] PASSED [ 24%]
[dspy-pytest] tests/clients/test_lm.py::test_reasoning_model_requirements[openai/gpt-5-mini] PASSED [ 24%]
[dspy-pytest] tests/clients/test_lm.py::test_gpt_5_chat_not_reasoning_model PASSED     [ 24%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_init_uses_lm_defaults_and_isolates_callback_list PASSED [ 25%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_forward_contract_defaults_to_legacy PASSED [ 25%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_forward_contract_accepts_explicit_values PASSED [ 25%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_forward_contract_rejects_unknown_values PASSED [ 25%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_validates_typed_lm_response PASSED [ 25%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_warns_when_inherited_legacy_forward_returns_lm_response PASSED [ 25%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_errors_when_explicit_legacy_forward_returns_lm_response PASSED [ 25%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_inherited_legacy_forward_returning_lm_response_errors_on_direct_call PASSED [ 25%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_default_call_keeps_legacy_outputs PASSED [ 25%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_experimental_call_returns_lm_response_through_legacy_bridge PASSED [ 25%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_explicit_lm_request_returns_lm_response_without_experimental PASSED [ 25%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_legacy_bridge_records_typed_history_and_usage_once PASSED [ 25%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_typed_forward_contract_uses_lm_request PASSED [ 25%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_typed_forward_contract_rejects_non_lm_response_at_call_time PASSED [ 26%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_request_call_rejects_mixed_inputs PASSED [ 26%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_system_user_and_assistant_turns[current_lm] PASSED [ 26%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_system_user_and_assistant_turns[typed_lm] PASSED [ 26%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_tool_call_transcripts[current_lm] PASSED [ 26%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_support_tool_call_transcripts[typed_lm] PASSED [ 26%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_can_reuse_lm_response_as_assistant_turn[current_lm] PASSED [ 26%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_experimental_direct_messages_can_reuse_lm_response_as_assistant_turn[typed_lm] PASSED [ 26%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_async_explicit_lm_request_returns_lm_response PASSED [ 26%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_tracks_usage_for_custom_subclasses PASSED [ 26%]
[dspy-pytest] tests/clients/test_lm.py::test_base_lm_copy_is_shallow_runtime_copy_with_isolated_dspy_state PASSED [ 26%]
[dspy-pytest] tests/clients/test_lm.py::test_dump_state PASSED                         [ 26%]
[dspy-pytest] tests/clients/test_lm.py::test_reasoning_model_dump_state_uses_constructor_max_tokens PASSED [ 26%]
[dspy-pytest] tests/clients/test_lm.py::test_dump_state_preserves_enabled_developer_role PASSED [ 27%]
[dspy-pytest] tests/clients/test_lm.py::test_dump_state_ignores_internal_class_marker_kwarg PASSED [ 27%]
[dspy-pytest] tests/clients/test_lm.py::test_load_state PASSED                         [ 27%]
[dspy-pytest] tests/clients/test_lm.py::test_reasoning_model_load_state_round_trips_canonical_state PASSED [ 27%]
[dspy-pytest] tests/clients/test_lm.py::test_reasoning_model_load_state_accepts_max_completion_tokens_alias PASSED [ 27%]
[dspy-pytest] tests/clients/test_lm.py::test_lm_load_state_forwards_allow_custom_lm_class PASSED [ 27%]
[dspy-pytest] tests/clients/test_lm.py::test_exponential_backoff_retry PASSED          [ 27%]
[dspy-pytest] tests/clients/test_lm.py::test_logprobs_included_when_requested PASSED   [ 27%]
[dspy-pytest] tests/clients/test_lm.py::test_async_lm_call PASSED                      [ 27%]
[dspy-pytest] tests/clients/test_lm.py::test_async_lm_call_with_cache PASSED           [ 27%]
[dspy-pytest] tests/clients/test_lm.py::test_lm_history_size_limit PASSED              [ 27%]
[dspy-pytest] tests/clients/test_lm.py::test_disable_history PASSED                    [ 27%]
[dspy-pytest] tests/clients/test_lm.py::test_responses_api PASSED                      [ 27%]
[dspy-pytest] tests/clients/test_lm.py::test_lm_replaces_system_with_developer_role PASSED [ 28%]
[dspy-pytest] tests/clients/test_lm.py::test_responses_api_tool_calls PASSED           [ 28%]
[dspy-pytest] tests/clients/test_lm.py::test_reasoning_effort_responses_api PASSED     [ 28%]
[dspy-pytest] tests/clients/test_lm.py::test_call_reasoning_model_with_chat_api PASSED [ 28%]
[dspy-pytest] tests/clients/test_lm.py::test_api_key_not_saved_in_json PASSED          [ 28%]
[dspy-pytest] tests/clients/test_lm.py::test_responses_api_converts_images_correctly PASSED [ 28%]
[dspy-pytest] tests/clients/test_lm.py::test_responses_api_converts_files_correctly PASSED [ 28%]
[dspy-pytest] tests/clients/test_lm.py::test_responses_api_preserves_multi_message_structure PASSED [ 28%]
[dspy-pytest] tests/clients/test_lm.py::test_responses_api_with_image_input PASSED     [ 28%]
[dspy-pytest] tests/clients/test_lm.py::test_responses_api_with_pydantic_model_input PASSED [ 28%]
[dspy-pytest] tests/clients/test_lm.py::test_responses_api_with_none_usage PASSED      [ 28%]
[dspy-pytest] tests/clients/test_lm.py::test_responses_api_with_none_usage_async PASSED [ 28%]
[dspy-pytest] tests/clients/test_lm.py::test_streaming_passes_headers_correctly PASSED [ 28%]
[dspy-pytest] tests/clients/test_lm_direct_live.py::test_live_openai_chat_direct_system_user_assistant_multiturn SKIPPED [ 29%]
[dspy-pytest] tests/clients/test_lm_direct_live.py::test_live_openai_chat_direct_tool_call_transcript SKIPPED [ 29%]
[dspy-pytest] tests/clients/test_lm_direct_live.py::test_live_openai_chat_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 29%]
[dspy-pytest] tests/clients/test_lm_direct_live.py::test_live_openai_responses_direct_system_user_assistant_multiturn SKIPPED [ 29%]
[dspy-pytest] tests/clients/test_lm_direct_live.py::test_live_openai_responses_direct_tool_call_transcript SKIPPED [ 29%]
[dspy-pytest] tests/clients/test_lm_direct_live.py::test_live_openai_responses_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 29%]
[dspy-pytest] tests/clients/test_lm_direct_live.py::test_live_anthropic_chat_direct_system_user_assistant_multiturn SKIPPED [ 29%]
[dspy-pytest] tests/clients/test_lm_direct_live.py::test_live_anthropic_chat_direct_tool_call_transcript SKIPPED [ 29%]
[dspy-pytest] tests/clients/test_lm_direct_live.py::test_live_anthropic_chat_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 29%]
[dspy-pytest] tests/clients/test_lm_direct_live.py::test_live_gemini_chat_direct_system_user_assistant_multiturn SKIPPED [ 29%]
[dspy-pytest] tests/clients/test_lm_direct_live.py::test_live_gemini_chat_direct_tool_call_transcript SKIPPED [ 29%]
[dspy-pytest] tests/clients/test_lm_direct_live.py::test_live_gemini_chat_direct_reuse_lm_response_as_assistant_turn SKIPPED [ 29%]
[dspy-pytest] tests/clients/test_lm_local.py::test_command_with_spaces_in_path PASSED  [ 29%]
[dspy-pytest] tests/clients/test_lm_local.py::test_command_construction_prevents_injection PASSED [ 30%]
[dspy-pytest] tests/clients/test_lm_local.py::test_command_is_list_not_string PASSED   [ 30%]
[dspy-pytest] tests/core/test_types.py::test_message_content_and_tool_calls_normalize_for_dspy_history_surface PASSED [ 30%]
[dspy-pytest] tests/core/test_types.py::test_tool_result_content_none_normalizes_to_empty_parts PASSED [ 30%]
[dspy-pytest] tests/core/test_types.py::test_audio_content_accepts_url_and_history_preserves_url PASSED [ 30%]
[dspy-pytest] tests/core/test_types.py::test_audio_content_defaults_null_format_to_wav PASSED [ 30%]
[dspy-pytest] tests/core/test_types.py::test_image_content_requires_mapping_with_url PASSED [ 30%]
[dspy-pytest] tests/core/test_types.py::test_video_data_round_trips_through_history_messages PASSED [ 30%]
[dspy-pytest] tests/core/test_types.py::test_document_source_url_stays_url_and_round_trips_through_history_messages PASSED [ 30%]
[dspy-pytest] tests/core/test_types.py::test_config_extensions_flatten_when_converted_to_legacy_kwargs PASSED [ 30%]
[dspy-pytest] tests/core/test_types.py::test_lm_kwargs_aliases_normalize_for_existing_dspy_lm_callers PASSED [ 30%]
[dspy-pytest] tests/core/test_types.py::test_nested_config_aliases_remain_supported_for_existing_interfaces PASSED [ 30%]
[dspy-pytest] tests/core/test_types.py::test_usage_normalizes_existing_user_visible_token_aliases PASSED [ 30%]
[dspy-pytest] tests/core/test_types.py::test_default_config_does_not_serialize_empty_stop_sequences PASSED [ 30%]
[dspy-pytest] tests/core/test_types.py::test_history_entry_exposes_existing_dict_style_properties PASSED [ 31%]
[dspy-pytest] tests/core/test_types.py::test_response_rejects_empty_outputs PASSED     [ 31%]
[dspy-pytest] tests/core/test_types.py::test_output_to_value_preserves_redacted_thinking_part PASSED [ 31%]
[dspy-pytest] tests/core/test_types.py::test_stream_event_indices_must_be_non_negative PASSED [ 31%]
[dspy-pytest] tests/core/test_types.py::test_stream_builder_rejects_sparse_output_indices PASSED [ 31%]
[dspy-pytest] tests/core/test_types.py::test_stream_builder_rejects_sparse_part_indices PASSED [ 31%]
[dspy-pytest] tests/core/test_types.py::test_stream_builder_rejects_delta_type_changes PASSED [ 31%]
[dspy-pytest] tests/core/test_types.py::test_stream_builder_rejects_incomplete_tool_call_arguments PASSED [ 31%]
[dspy-pytest] tests/datasets/test_dataset.py::test_input_keys SKIPPED (need --extra
[dspy-pytest] option to run)                                                           [ 31%]
[dspy-pytest] tests/docs/test_mkdocs_links.py::test_nav_files_exist PASSED             [ 31%]
[dspy-pytest] tests/evaluate/test_auto_evaluation.py::test_semantic_f1_returns_prediction_without_trace PASSED [ 31%]
[dspy-pytest] tests/evaluate/test_auto_evaluation.py::test_semantic_f1_returns_prediction_with_trace PASSED [ 31%]
[dspy-pytest] tests/evaluate/test_auto_evaluation.py::test_semantic_f1_score_value PASSED [ 31%]
[dspy-pytest] tests/evaluate/test_auto_evaluation.py::test_complete_and_grounded_returns_prediction_without_trace PASSED [ 32%]
[dspy-pytest] tests/evaluate/test_auto_evaluation.py::test_complete_and_grounded_returns_prediction_with_trace PASSED [ 32%]
[dspy-pytest] tests/evaluate/test_auto_evaluation.py::test_semantic_f1_prediction_can_be_compared PASSED [ 32%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_initialization PASSED     [ 32%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_call PASSED               [ 32%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_single_thread_runs_in_main_thread PASSED [ 32%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_construct_result_df SKIPPED (need
[dspy-pytest] --extra option to run)                                                   [ 32%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_multithread_evaluate_call PASSED   [ 32%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_multi_thread_evaluate_call_cancelled PASSED [ 32%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_call_wrong_answer PASSED  [ 32%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example0] SKIPPED [ 32%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example1] SKIPPED [ 32%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example2] SKIPPED [ 32%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-True-program_with_example3] SKIPPED [ 33%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example0] SKIPPED [ 33%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example1] SKIPPED [ 33%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example2] SKIPPED [ 33%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-False-program_with_example3] SKIPPED [ 33%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example0] SKIPPED [ 33%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example1] SKIPPED [ 33%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example2] SKIPPED [ 33%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[True-1-program_with_example3] SKIPPED [ 33%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example0] SKIPPED [ 33%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example1] SKIPPED [ 33%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example2] SKIPPED [ 33%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-True-program_with_example3] SKIPPED [ 33%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example0] SKIPPED [ 34%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example1] SKIPPED [ 34%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example2] SKIPPED [ 34%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-False-program_with_example3] SKIPPED [ 34%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example0] SKIPPED [ 34%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example1] SKIPPED [ 34%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example2] SKIPPED [ 34%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_display_table[False-1-program_with_example3] SKIPPED [ 34%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_callback PASSED           [ 34%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluation_result_repr PASSED      [ 34%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_save_as_json_with_history PASSED [ 34%]
[dspy-pytest] tests/evaluate/test_evaluate.py::test_evaluate_save_as_csv_with_history PASSED [ 34%]
[dspy-pytest] tests/evaluate/test_metrics.py::test_answer_exact_match_string PASSED    [ 34%]
[dspy-pytest] tests/evaluate/test_metrics.py::test_answer_exact_match_list PASSED      [ 35%]
[dspy-pytest] tests/evaluate/test_metrics.py::test_answer_exact_match_no_match PASSED  [ 35%]
[dspy-pytest] tests/metadata/test_metadata.py::test_metadata PASSED                    [ 35%]
[dspy-pytest] tests/predict/test_aggregation.py::test_majority_with_prediction PASSED  [ 35%]
[dspy-pytest] tests/predict/test_aggregation.py::test_majority_with_completions PASSED [ 35%]
[dspy-pytest] tests/predict/test_aggregation.py::test_majority_with_list PASSED        [ 35%]
[dspy-pytest] tests/predict/test_aggregation.py::test_majority_with_normalize PASSED   [ 35%]
[dspy-pytest] tests/predict/test_aggregation.py::test_majority_with_field PASSED       [ 35%]
[dspy-pytest] tests/predict/test_aggregation.py::test_majority_with_no_majority PASSED [ 35%]
[dspy-pytest] tests/predict/test_best_of_n.py::test_refine_forward_success_first_attempt PASSED [ 35%]
[dspy-pytest] tests/predict/test_best_of_n.py::test_refine_module_default_fail_count PASSED [ 35%]
[dspy-pytest] tests/predict/test_best_of_n.py::test_refine_module_custom_fail_count PASSED [ 35%]
[dspy-pytest] tests/predict/test_chain_of_thought.py::test_initialization_with_string_signature PASSED [ 35%]
[dspy-pytest] tests/predict/test_chain_of_thought.py::test_async_chain_of_thought PASSED [ 36%]
[dspy-pytest] tests/predict/test_chain_of_thought.py::test_chain_of_thought_with_native_reasoning PASSED [ 36%]
[dspy-pytest] tests/predict/test_chain_of_thought.py::test_chain_of_thought_with_manual_reasoning PASSED [ 36%]
[dspy-pytest] tests/predict/test_code_act.py::test_codeact_code_generation SKIPPED     [ 36%]
[dspy-pytest] tests/predict/test_code_act.py::test_codeact_support_multiple_fields SKIPPED [ 36%]
[dspy-pytest] tests/predict/test_code_act.py::test_codeact_code_parse_failure SKIPPED  [ 36%]
[dspy-pytest] tests/predict/test_code_act.py::test_codeact_code_execution_failure SKIPPED [ 36%]
[dspy-pytest] tests/predict/test_code_act.py::test_codeact_tool_validation SKIPPED     [ 36%]
[dspy-pytest] tests/predict/test_knn.py::test_knn_initialization SKIPPED (need --extra
[dspy-pytest] option to run)                                                           [ 36%]
[dspy-pytest] tests/predict/test_knn.py::test_knn_query SKIPPED (need --extra option
[dspy-pytest] to run)                                                                  [ 36%]
[dspy-pytest] tests/predict/test_knn.py::test_knn_query_specificity SKIPPED (need
[dspy-pytest] --extra option to run)                                                   [ 36%]
[dspy-pytest] tests/predict/test_multi_chain_comparison.py::test_basic_example PASSED  [ 36%]
[dspy-pytest] tests/predict/test_parallel.py::test_parallel_module PASSED              [ 36%]
[dspy-pytest] tests/predict/test_parallel.py::test_batch_module PASSED                 [ 37%]
[dspy-pytest] tests/predict/test_parallel.py::test_nested_parallel_module PASSED       [ 37%]
[dspy-pytest] tests/predict/test_parallel.py::test_nested_batch_method PASSED          [ 37%]
[dspy-pytest] tests/predict/test_parallel.py::test_batch_with_failed_examples PASSED   [ 37%]
[dspy-pytest] tests/predict/test_parallel.py::test_parallel_timeout_and_straggler_limit_params PASSED [ 37%]
[dspy-pytest] tests/predict/test_parallel.py::test_batch_timeout_and_straggler_limit_params PASSED [ 37%]
[dspy-pytest] tests/predict/test_predict.py::test_initialization_with_string_signature PASSED [ 37%]
[dspy-pytest] tests/predict/test_predict.py::test_reset_method PASSED                  [ 37%]
[dspy-pytest] tests/predict/test_predict.py::test_lm_after_dump_and_load_state PASSED  [ 37%]
[dspy-pytest] tests/predict/test_predict.py::test_base_lm_dump_state_ignores_internal_class_marker_kwarg PASSED [ 37%]
[dspy-pytest] tests/predict/test_predict.py::test_legacy_lm_state_without_class_marker_loads_as_lm PASSED [ 37%]
[dspy-pytest] tests/predict/test_predict.py::test_custom_lm_load_state_requires_trusted_opt_in PASSED [ 37%]
[dspy-pytest] tests/predict/test_predict.py::test_nested_custom_lm_class_path_loads_for_trusted_state PASSED [ 37%]
[dspy-pytest] tests/predict/test_predict.py::test_call_method PASSED                   [ 38%]
[dspy-pytest] tests/predict/test_predict.py::test_instructions_after_dump_and_load_state PASSED [ 38%]
[dspy-pytest] tests/predict/test_predict.py::test_demos_after_dump_and_load_state PASSED [ 38%]
[dspy-pytest] tests/predict/test_predict.py::test_typed_demos_after_dump_and_load_state PASSED [ 38%]
[dspy-pytest] tests/predict/test_predict.py::test_signature_fields_after_dump_and_load_state PASSED [ 38%]
[dspy-pytest] tests/predict/test_predict.py::test_lm_field_after_dump_and_load_state[model.json] PASSED [ 38%]
[dspy-pytest] tests/predict/test_predict.py::test_lm_field_after_dump_and_load_state[model.pkl] PASSED [ 38%]
[dspy-pytest] tests/predict/test_predict.py::test_load_ignores_serialized_endpoint_override_by_default[api_base] PASSED [ 38%]
[dspy-pytest] tests/predict/test_predict.py::test_load_ignores_serialized_endpoint_override_by_default[base_url] PASSED [ 38%]
[dspy-pytest] tests/predict/test_predict.py::test_load_allows_serialized_endpoint_override_with_opt_in[api_base] PASSED [ 38%]
[dspy-pytest] tests/predict/test_predict.py::test_load_allows_serialized_endpoint_override_with_opt_in[base_url] PASSED [ 38%]
[dspy-pytest] tests/predict/test_predict.py::test_load_state_ignores_serialized_endpoint_override_by_default[api_base] PASSED [ 38%]
[dspy-pytest] tests/predict/test_predict.py::test_load_state_ignores_serialized_endpoint_override_by_default[base_url] PASSED [ 38%]
[dspy-pytest] tests/predict/test_predict.py::test_load_state_allows_serialized_endpoint_override_with_opt_in[api_base] PASSED [ 38%]
[dspy-pytest] tests/predict/test_predict.py::test_load_state_allows_serialized_endpoint_override_with_opt_in[base_url] PASSED [ 39%]
[dspy-pytest] tests/predict/test_predict.py::test_load_state_ignores_serialized_model_list_endpoint_override_by_default PASSED [ 39%]
[dspy-pytest] tests/predict/test_predict.py::test_load_prevents_serialized_endpoint_override_reaching_litellm[api_base] PASSED [ 39%]
[dspy-pytest] tests/predict/test_predict.py::test_load_prevents_serialized_endpoint_override_reaching_litellm[base_url] PASSED [ 39%]
[dspy-pytest] tests/predict/test_predict.py::test_load_blocks_serialized_model_list_unless_opted_in PASSED [ 39%]
[dspy-pytest] tests/predict/test_predict.py::test_load_uses_env_api_key_without_honoring_serialized_endpoint_override PASSED [ 39%]
[dspy-pytest] tests/predict/test_predict.py::test_forward_method PASSED                [ 39%]
[dspy-pytest] tests/predict/test_predict.py::test_forward_method2 PASSED               [ 39%]
[dspy-pytest] tests/predict/test_predict.py::test_config_management PASSED             [ 39%]
[dspy-pytest] tests/predict/test_predict.py::test_multi_output PASSED                  [ 39%]
[dspy-pytest] tests/predict/test_predict.py::test_multi_output2 PASSED                 [ 39%]
[dspy-pytest] tests/predict/test_predict.py::test_datetime_inputs_and_outputs PASSED   [ 39%]
[dspy-pytest] tests/predict/test_predict.py::test_explicitly_valued_enum_inputs_and_outputs PASSED [ 39%]
[dspy-pytest] tests/predict/test_predict.py::test_enum_inputs_and_outputs_with_shared_names_and_values PASSED [ 40%]
[dspy-pytest] tests/predict/test_predict.py::test_auto_valued_enum_inputs_and_outputs PASSED [ 40%]
[dspy-pytest] tests/predict/test_predict.py::test_named_predictors PASSED              [ 40%]
[dspy-pytest] tests/predict/test_predict.py::test_output_only PASSED                   [ 40%]
[dspy-pytest] tests/predict/test_predict.py::test_load_state_chaining PASSED           [ 40%]
[dspy-pytest] tests/predict/test_predict.py::test_call_predict_with_chat_history[chat] PASSED [ 40%]
[dspy-pytest] tests/predict/test_predict.py::test_call_predict_with_chat_history[json] PASSED [ 40%]
[dspy-pytest] tests/predict/test_predict.py::test_lm_usage PASSED                      [ 40%]
[dspy-pytest] tests/predict/test_predict.py::test_lm_usage_with_parallel PASSED        [ 40%]
[dspy-pytest] tests/predict/test_predict.py::test_lm_usage_with_async PASSED           [ 40%]
[dspy-pytest] tests/predict/test_predict.py::test_positional_arguments PASSED          [ 40%]
[dspy-pytest] tests/predict/test_predict.py::test_error_message_on_invalid_lm_setup PASSED [ 40%]
[dspy-pytest] tests/predict/test_predict.py::test_field_constraints[chat] PASSED       [ 40%]
[dspy-pytest] tests/predict/test_predict.py::test_field_constraints[json] PASSED       [ 41%]
[dspy-pytest] tests/predict/test_predict.py::test_async_predict PASSED                 [ 41%]
[dspy-pytest] tests/predict/test_predict.py::test_predicted_outputs_piped_from_predict_to_lm_call PASSED [ 41%]
[dspy-pytest] tests/predict/test_predict.py::test_dump_state_pydantic_non_primitive_types PASSED [ 41%]
[dspy-pytest] tests/predict/test_predict.py::test_trace_size_limit PASSED              [ 41%]
[dspy-pytest] tests/predict/test_predict.py::test_disable_trace PASSED                 [ 41%]
[dspy-pytest] tests/predict/test_predict.py::test_per_module_history_size_limit PASSED [ 41%]
[dspy-pytest] tests/predict/test_predict.py::test_per_module_history_disabled PASSED   [ 41%]
[dspy-pytest] tests/predict/test_predict.py::test_input_field_default_value PASSED     [ 41%]
[dspy-pytest] tests/predict/test_predict.py::test_extra_fields_warning PASSED          [ 41%]
[dspy-pytest] tests/predict/test_predict.py::test_missing_optional_input_field_no_warning PASSED [ 41%]
[dspy-pytest] tests/predict/test_predict.py::test_missing_required_input_field_still_warns PASSED [ 41%]
[dspy-pytest] tests/predict/test_predict.py::test_warning_images PASSED                [ 41%]
[dspy-pytest] tests/predict/test_predict.py::test_type_mismatch_warning PASSED         [ 42%]
[dspy-pytest] tests/predict/test_predict.py::test_correct_types_no_warning PASSED      [ 42%]
[dspy-pytest] tests/predict/test_predict.py::test_list_type_validation PASSED          [ 42%]
[dspy-pytest] tests/predict/test_predict.py::test_literal_type_validation PASSED       [ 42%]
[dspy-pytest] tests/predict/test_predict.py::test_literal_union_type_validation PASSED [ 42%]
[dspy-pytest] tests/predict/test_predict.py::test_list_string PASSED                   [ 42%]
[dspy-pytest] tests/predict/test_predict.py::test_nested_list_type_validation PASSED   [ 42%]
[dspy-pytest] tests/predict/test_predict.py::test_nested_dict_type_validation PASSED   [ 42%]
[dspy-pytest] tests/predict/test_predict.py::test_nested_tuple_type_validation PASSED  [ 42%]
[dspy-pytest] tests/predict/test_predict.py::test_literal_type_validation_string_signature PASSED [ 42%]
[dspy-pytest] tests/predict/test_predict.py::test_list_type_validation_string_signature PASSED [ 42%]
[dspy-pytest] tests/predict/test_predict.py::test_dict_type_validation_string_signature PASSED [ 42%]
[dspy-pytest] tests/predict/test_predict.py::test_tuple_type_validation_string_signature PASSED [ 42%]
[dspy-pytest] tests/predict/test_predict.py::test_union_type_validation_string_signature PASSED [ 43%]
[dspy-pytest] tests/predict/test_predict.py::test_basic_types_string_signature[False] PASSED [ 43%]
[dspy-pytest] tests/predict/test_predict.py::test_basic_types_string_signature[True] PASSED [ 43%]
[dspy-pytest] tests/predict/test_predict.py::test_untyped_string_signature PASSED      [ 43%]
[dspy-pytest] tests/predict/test_predict.py::test_untyped_class_signature PASSED       [ 43%]
[dspy-pytest] tests/predict/test_predict.py::test_string_to_list_signature PASSED      [ 43%]
[dspy-pytest] tests/predict/test_predict.py::test_custom_signature_types[False] PASSED [ 43%]
[dspy-pytest] tests/predict/test_predict.py::test_custom_signature_types[True] PASSED  [ 43%]
[dspy-pytest] tests/predict/test_program_of_thought.py::test_pot_code_generation SKIPPED [ 43%]
[dspy-pytest] tests/predict/test_program_of_thought.py::test_old_style_pot SKIPPED     [ 43%]
[dspy-pytest] tests/predict/test_program_of_thought.py::test_pot_support_multiple_fields SKIPPED [ 43%]
[dspy-pytest] tests/predict/test_program_of_thought.py::test_pot_code_generation_with_one_error SKIPPED [ 43%]
[dspy-pytest] tests/predict/test_program_of_thought.py::test_pot_code_generation_persistent_errors SKIPPED [ 43%]
[dspy-pytest] tests/predict/test_program_of_thought.py::test_pot_code_parse_error PASSED [ 44%]
[dspy-pytest] tests/predict/test_react.py::test_tool_observation_preserves_custom_type SKIPPED [ 44%]
[dspy-pytest] tests/predict/test_react.py::test_tool_calling_with_pydantic_args PASSED [ 44%]
[dspy-pytest] tests/predict/test_react.py::test_react_with_tools_skips_native_response_issubclass_for_generic_alias PASSED [ 44%]
[dspy-pytest] tests/predict/test_react.py::test_tool_calling_without_typehint PASSED   [ 44%]
[dspy-pytest] tests/predict/test_react.py::test_trajectory_truncation PASSED           [ 44%]
[dspy-pytest] tests/predict/test_react.py::test_context_window_exceeded_after_retries PASSED [ 44%]
[dspy-pytest] tests/predict/test_react.py::test_error_retry PASSED                     [ 44%]
[dspy-pytest] tests/predict/test_react.py::test_async_tool_calling_with_pydantic_args PASSED [ 44%]
[dspy-pytest] tests/predict/test_react.py::test_async_error_retry PASSED               [ 44%]
[dspy-pytest] tests/predict/test_react_v2.py::test_react_v2_submit_tool_returns_original_output_fields PASSED [ 44%]
[dspy-pytest] tests/predict/test_react_v2.py::test_react_v2_text_mock_lm_loop_records_inputs_once PASSED [ 44%]
[dspy-pytest] tests/predict/test_react_v2.py::test_react_v2_continuation_omits_missing_original_inputs PASSED [ 44%]
[dspy-pytest] tests/predict/test_react_v2.py::test_react_v2_text_mode_accepts_top_level_tool_arguments PASSED [ 45%]
[dspy-pytest] tests/predict/test_react_v2.py::test_react_v2_text_mode_accepts_wrapped_submit_arguments PASSED [ 45%]
[dspy-pytest] tests/predict/test_react_v2.py::test_react_v2_unknown_tool_observation_can_continue PASSED [ 45%]
[dspy-pytest] tests/predict/test_react_v2.py::test_react_v2_accepts_serialized_history_input PASSED [ 45%]
[dspy-pytest] tests/predict/test_react_v2.py::test_react_v2_forced_submit_on_empty_tool_calls PASSED [ 45%]
[dspy-pytest] tests/predict/test_react_v2.py::test_react_v2_native_tool_loop_replays_tool_result_with_provider_id PASSED [ 45%]
[dspy-pytest] tests/predict/test_react_v2.py::test_react_v2_native_parallel_tool_calls_are_requested_and_replayed PASSED [ 45%]
[dspy-pytest] tests/predict/test_refine.py::test_refine_forward_success_first_attempt PASSED [ 45%]
[dspy-pytest] tests/predict/test_refine.py::test_refine_module_default_fail_count PASSED [ 45%]
[dspy-pytest] tests/predict/test_refine.py::test_refine_module_custom_fail_count PASSED [ 45%]
[dspy-pytest] tests/predict/test_rlm.py::TestMockInterpreter::test_scripted_responses PASSED [ 45%]
[dspy-pytest] tests/predict/test_rlm.py::TestMockInterpreter::test_returns_final_output_result PASSED [ 45%]
[dspy-pytest] tests/predict/test_rlm.py::TestMockInterpreter::test_raises_exception_from_responses PASSED [ 45%]
[dspy-pytest] tests/predict/test_rlm.py::TestMockInterpreter::test_records_call_history PASSED [ 46%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMInitialization::test_basic_initialization PASSED [ 46%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMInitialization::test_custom_signature PASSED [ 46%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMInitialization::test_custom_tools PASSED [ 46%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_invalid_identifier[invalid-name] PASSED [ 46%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_invalid_identifier[123start] PASSED [ 46%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_reserved_names[llm_query] PASSED [ 46%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_reserved_names[SUBMIT] PASSED [ 46%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_reserved_names[print] PASSED [ 46%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_not_callable[not a function] PASSED [ 46%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMInitialization::test_tool_validation_not_callable[123] PASSED [ 46%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMInitialization::test_tools_dict_rejected PASSED [ 46%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMInitialization::test_optional_parameters PASSED [ 46%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMInitialization::test_forward_validates_required_inputs PASSED [ 46%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMInitialization::test_batched_query_errors_have_clear_markers PASSED [ 47%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMInitialization::test_tools_call_counter_is_thread_safe PASSED [ 47%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```python\nprint(1)\n```-print(1)] PASSED [ 47%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```py\nx = 1\nprint(x)\n```-x = 1\nprint(x)] PASSED [ 47%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```\nprint('no lang')\n```-print('no lang')] PASSED [ 47%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[not fenced code-not fenced code] PASSED [ 47%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[I'll inspect first.\n```python\nprint('hello')\n```\nThen I will submit.-print('hello')] PASSED [ 47%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```python\nprint(1)\n```\nsome trailing text-print(1)] PASSED [ 47%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```python\nprint('oops')-print('oops')] PASSED [ 47%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```\n```python\nprint(1)\n```\n```-print(1)] PASSED [ 47%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences[```\n```\nprint(2)\n```\n```-print(2)] PASSED [ 47%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMCodeFenceParsing::test_strip_code_fences_rejects_non_python_lang PASSED [ 47%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMFormatting::test_format_history PASSED [ 47%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMFormatting::test_format_history_empty PASSED [ 48%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMFormatting::test_action_signature_has_iteration_field PASSED [ 48%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMFormatting::test_format_output PASSED  [ 48%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMFormatting::test_format_output_empty PASSED [ 48%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMFormatting::test_format_output_passthrough PASSED [ 48%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMFormatting::test_format_variable_info_string PASSED [ 48%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMFormatting::test_format_variable_info_dict PASSED [ 48%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMFormatting::test_build_variables_multiple PASSED [ 48%]
[dspy-pytest] tests/predict/test_rlm.py::TestREPLTypes::test_repl_history_immutability PASSED [ 48%]
[dspy-pytest] tests/predict/test_rlm.py::TestREPLTypes::test_repl_history_len_iter_bool PASSED [ 48%]
[dspy-pytest] tests/predict/test_rlm.py::TestREPLTypes::test_repl_entry_format PASSED  [ 48%]
[dspy-pytest] tests/predict/test_rlm.py::TestREPLTypes::test_repl_entry_format_truncation PASSED [ 48%]
[dspy-pytest] tests/predict/test_rlm.py::TestREPLTypes::test_repl_entry_format_no_truncation PASSED [ 48%]
[dspy-pytest] tests/predict/test_rlm.py::TestREPLTypes::test_repl_history_threads_max_output_chars PASSED [ 49%]
[dspy-pytest] tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_from_value PASSED [ 49%]
[dspy-pytest] tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_truncation PASSED [ 49%]
[dspy-pytest] tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_with_field_info PASSED [ 49%]
[dspy-pytest] tests/predict/test_rlm.py::TestREPLTypes::test_repl_variable_without_field_info PASSED [ 49%]
[dspy-pytest] tests/predict/test_rlm.py::TestREPLTypes::test_build_variables_includes_field_metadata PASSED [ 49%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMCallMethod::test_call_is_alias_for_forward PASSED [ 49%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMMaxIterationsFallback::test_max_iters_triggers_extract PASSED [ 49%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMToolExceptions::test_tool_exception_returns_error_in_output PASSED [ 49%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMToolExceptions::test_runtime_error_history_uses_stripped_code PASSED [ 49%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMToolExceptions::test_syntax_error_from_execute_is_recoverable PASSED [ 49%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMToolExceptions::test_syntax_error_from_strip_code_fences_is_recoverable PASSED [ 49%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMDynamicSignature::test_action_signature_structure PASSED [ 49%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMDynamicSignature::test_extract_signature_structure PASSED [ 50%]
[dspy-pytest] tests/predict/test_rlm.py::TestPythonInterpreter::test_start_prewarms_sandbox SKIPPED [ 50%]
[dspy-pytest] tests/predict/test_rlm.py::TestPythonInterpreter::test_start_is_idempotent SKIPPED [ 50%]
[dspy-pytest] tests/predict/test_rlm.py::TestPythonInterpreter::test_basic_execution SKIPPED [ 50%]
[dspy-pytest] tests/predict/test_rlm.py::TestPythonInterpreter::test_variable_injection SKIPPED [ 50%]
[dspy-pytest] tests/predict/test_rlm.py::TestPythonInterpreter::test_variable_injection_with_none_values SKIPPED [ 50%]
[dspy-pytest] tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_call_kwargs SKIPPED [ 50%]
[dspy-pytest] tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_call_positional SKIPPED [ 50%]
[dspy-pytest] tests/predict/test_rlm.py::TestPythonInterpreter::test_multiple_tools SKIPPED [ 50%]
[dspy-pytest] tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_returns_list SKIPPED [ 50%]
[dspy-pytest] tests/predict/test_rlm.py::TestPythonInterpreter::test_tool_returns_dict SKIPPED [ 50%]
[dspy-pytest] tests/predict/test_rlm.py::TestPythonInterpreter::test_state_persists SKIPPED [ 50%]
[dspy-pytest] tests/predict/test_rlm.py::TestPythonInterpreter::test_syntax_error SKIPPED [ 50%]
[dspy-pytest] tests/predict/test_rlm.py::TestPythonInterpreter::test_runtime_error SKIPPED [ 51%]
[dspy-pytest] tests/predict/test_rlm.py::TestSandboxSecurity::test_no_network_access SKIPPED [ 51%]
[dspy-pytest] tests/predict/test_rlm.py::TestSandboxSecurity::test_imports_work SKIPPED [ 51%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMAsyncMock::test_aforward_basic PASSED  [ 51%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMAsyncMock::test_aforward_int_output_mock PASSED [ 51%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMAsyncMock::test_aforward_multi_iteration_mock PASSED [ 51%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[count-int-42-SUBMIT(42)-42] PASSED [ 51%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[score-float-3.14-SUBMIT(3.14)-3.14] PASSED [ 51%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[valid-bool-True-SUBMIT(True)-True] PASSED [ 51%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[numbers-list[int]-final_value3-SUBMIT([1, 2, 3])-expected3] PASSED [ 51%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_coercion[answer-Literal['yes', 'no']-yes-SUBMIT("yes")-yes] PASSED [ 51%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMTypeCoercionMock::test_type_error_retries PASSED [ 51%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[count-int-SUBMIT(42)-42-int] SKIPPED [ 51%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[score-float-SUBMIT(3.14)-3.14-float] SKIPPED [ 52%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[valid-bool-SUBMIT(True)-True-bool] SKIPPED [ 52%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[numbers-list[int]-SUBMIT([1, 2, 3])-expected3-list] SKIPPED [ 52%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[data-dict[str, str]-SUBMIT({"key": "value"})-expected4-dict] SKIPPED [ 52%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMTypeCoercion::test_type_coercion[answer-Literal['yes', 'no']-SUBMIT("yes")-yes-str] SKIPPED [ 52%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMTypeCoercion::test_submit_extracts_typed_value SKIPPED [ 52%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_final_kwargs SKIPPED [ 52%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_final_positional SKIPPED [ 52%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_three_fields SKIPPED [ 52%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_final_missing_field_errors SKIPPED [ 52%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_submit_vars SKIPPED [ 52%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMMultipleOutputs::test_multi_output_type_coercion SKIPPED [ 52%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMWithDummyLM::test_simple_computation_e2e SKIPPED [ 52%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMWithDummyLM::test_multi_turn_computation_e2e SKIPPED [ 53%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMWithDummyLM::test_with_input_variables_e2e SKIPPED [ 53%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMWithDummyLM::test_with_tool_e2e SKIPPED [ 53%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMWithDummyLM::test_aforward_simple_computation_e2e SKIPPED [ 53%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMWithDummyLM::test_aforward_multi_turn_e2e SKIPPED [ 53%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMWithDummyLM::test_aforward_with_input_variables_e2e SKIPPED [ 53%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMIntegration::test_simple_computation SKIPPED [ 53%]
[dspy-pytest] tests/predict/test_rlm.py::TestRLMIntegration::test_with_llm_query SKIPPED [ 53%]
[dspy-pytest] tests/predict/test_rlm.py::TestBuildVariablesWithSerializable::test_serializable_uses_build_repl_variable PASSED [ 53%]
[dspy-pytest] tests/predict/test_rlm.py::TestBuildVariablesWithSerializable::test_regular_values_unchanged PASSED [ 53%]
[dspy-pytest] tests/predict/test_rlm.py::TestPrepareSerializableVars::test_separates_serializable_from_regular PASSED [ 53%]
[dspy-pytest] tests/predict/test_rlm.py::TestPrepareSerializableVars::test_no_serializable_returns_all PASSED [ 53%]
[dspy-pytest] tests/predict/test_rlm.py::TestPrepareSerializableVars::test_binary_payload_uses_base64_transport PASSED [ 53%]
[dspy-pytest] tests/predict/test_rlm.py::TestPrepareSerializableVars::test_large_payload_not_inlined_in_code PASSED [ 53%]
[dspy-pytest] tests/predict/test_rlm.py::TestPrepareSerializableVars::test_forward_with_serializable PASSED [ 54%]
[dspy-pytest] tests/predict/test_rlm.py::TestLargeSerializableRoundTrip::test_large_payload_round_trips_through_real_sandbox SKIPPED [ 54%]
[dspy-pytest] tests/primitives/test_base_module.py::test_deepcopy_basic PASSED         [ 54%]
[dspy-pytest] tests/primitives/test_base_module.py::test_deepcopy_with_uncopyable_modules PASSED [ 54%]
[dspy-pytest] tests/primitives/test_base_module.py::test_deepcopy_with_nested_modules PASSED [ 54%]
[dspy-pytest] tests/primitives/test_base_module.py::test_save_and_load_with_json PASSED [ 54%]
[dspy-pytest] tests/primitives/test_base_module.py::test_save_and_load_with_pkl SKIPPED [ 54%]
[dspy-pytest] tests/primitives/test_base_module.py::test_save_with_extra_modules PASSED [ 54%]
[dspy-pytest] tests/primitives/test_base_module.py::test_load_with_version_mismatch PASSED [ 54%]
[dspy-pytest] tests/primitives/test_base_module.py::test_single_module_call_with_usage_tracker SKIPPED [ 54%]
[dspy-pytest] tests/primitives/test_base_module.py::test_multi_module_call_with_usage_tracker SKIPPED [ 54%]
[dspy-pytest] tests/primitives/test_base_module.py::test_usage_tracker_in_parallel SKIPPED [ 54%]
[dspy-pytest] tests/primitives/test_base_module.py::test_usage_tracker_async_parallel PASSED [ 54%]
[dspy-pytest] tests/primitives/test_base_module.py::test_usage_tracker_no_side_effect PASSED [ 55%]
[dspy-pytest] tests/primitives/test_base_module.py::test_module_history PASSED         [ 55%]
[dspy-pytest] tests/primitives/test_base_module.py::test_module_history_with_concurrency PASSED [ 55%]
[dspy-pytest] tests/primitives/test_base_module.py::test_module_history_async PASSED   [ 55%]
[dspy-pytest] tests/primitives/test_base_module.py::test_forward_direct_call_warning PASSED [ 55%]
[dspy-pytest] tests/primitives/test_base_module.py::test_forward_through_call_no_warning PASSED [ 55%]
[dspy-pytest] tests/primitives/test_example.py::test_example_initialization PASSED     [ 55%]
[dspy-pytest] tests/primitives/test_example.py::test_example_initialization_from_base PASSED [ 55%]
[dspy-pytest] tests/primitives/test_example.py::test_example_initialization_from_dict PASSED [ 55%]
[dspy-pytest] tests/primitives/test_example.py::test_example_set_get_item PASSED       [ 55%]
[dspy-pytest] tests/primitives/test_example.py::test_example_attribute_access PASSED   [ 55%]
[dspy-pytest] tests/primitives/test_example.py::test_example_deletion PASSED           [ 55%]
[dspy-pytest] tests/primitives/test_example.py::test_example_len PASSED                [ 55%]
[dspy-pytest] tests/primitives/test_example.py::test_example_repr_str_img PASSED       [ 56%]
[dspy-pytest] tests/primitives/test_example.py::test_example_repr_str PASSED           [ 56%]
[dspy-pytest] tests/primitives/test_example.py::test_example_eq PASSED                 [ 56%]
[dspy-pytest] tests/primitives/test_example.py::test_example_hash PASSED               [ 56%]
[dspy-pytest] tests/primitives/test_example.py::test_example_keys_values_items PASSED  [ 56%]
[dspy-pytest] tests/primitives/test_example.py::test_example_get PASSED                [ 56%]
[dspy-pytest] tests/primitives/test_example.py::test_example_with_inputs PASSED        [ 56%]
[dspy-pytest] tests/primitives/test_example.py::test_example_inputs_labels PASSED      [ 56%]
[dspy-pytest] tests/primitives/test_example.py::test_example_copy_without PASSED       [ 56%]
[dspy-pytest] tests/primitives/test_example.py::test_example_to_dict PASSED            [ 56%]
[dspy-pytest] tests/primitives/test_example.py::test_example_to_dict_with_history PASSED [ 56%]
[dspy-pytest] tests/primitives/test_module.py::test_module_initialization PASSED       [ 56%]
[dspy-pytest] tests/primitives/test_module.py::test_named_predictors PASSED            [ 56%]
[dspy-pytest] tests/primitives/test_module.py::test_predictors PASSED                  [ 57%]
[dspy-pytest] tests/primitives/test_module.py::test_forward PASSED                     [ 57%]
[dspy-pytest] tests/primitives/test_module.py::test_nested_named_predictors PASSED     [ 57%]
[dspy-pytest] tests/primitives/test_module.py::test_empty_module PASSED                [ 57%]
[dspy-pytest] tests/primitives/test_module.py::test_single_level PASSED                [ 57%]
[dspy-pytest] tests/primitives/test_module.py::test_multiple_levels PASSED             [ 57%]
[dspy-pytest] tests/primitives/test_module.py::test_multiple_sub_modules PASSED        [ 57%]
[dspy-pytest] tests/primitives/test_module.py::test_non_base_module_attributes PASSED  [ 57%]
[dspy-pytest] tests/primitives/test_module.py::test_complex_module_traversal PASSED    [ 57%]
[dspy-pytest] tests/primitives/test_module.py::test_complex_module_traversal_with_same_module PASSED [ 57%]
[dspy-pytest] tests/primitives/test_module.py::test_complex_module_set_attribute_by_name PASSED [ 57%]
[dspy-pytest] tests/primitives/test_module.py::test_named_parameters_duplicate_references PASSED [ 57%]
[dspy-pytest] tests/primitives/test_module.py::test_load_dspy_program_cross_version PASSED [ 57%]
[dspy-pytest] tests/primitives/test_module.py::test_load_state_is_transactional PASSED [ 58%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_execute_simple_code SKIPPED [ 58%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_import SKIPPED (need
[dspy-pytest] --deno option to run)                                                    [ 58%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_user_variable_definitions SKIPPED [ 58%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_rejects_python_keywords_as_variable_names SKIPPED [ 58%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_failure_syntax_error SKIPPED [ 58%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_failure_zero_division SKIPPED [ 58%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_exception_args SKIPPED [ 58%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_submit_with_list SKIPPED [ 58%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_enable_env_vars_flag SKIPPED [ 58%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_read_file_access_control SKIPPED [ 58%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_enable_write_flag SKIPPED [ 58%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_enable_net_flag SKIPPED [ 58%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_interpreter_security_filesystem_access SKIPPED [ 59%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_tools_dict_is_copied SKIPPED [ 59%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_serialize_tuple SKIPPED [ 59%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_serialize_set SKIPPED  [ 59%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_serialize_set_mixed_types SKIPPED [ 59%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_deno_command_dict_raises_type_error SKIPPED [ 59%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_tool_with_typed_signature SKIPPED [ 59%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_tool_positional_args SKIPPED [ 59%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_tool_keyword_args SKIPPED [ 59%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_tool_default_args SKIPPED [ 59%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_tools_re_register_after_process_restart SKIPPED [ 59%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_mounts_replay_after_process_restart SKIPPED [ 59%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_tool_all_positional_args SKIPPED [ 59%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_tool_error_surfaces_as_runtime_error SKIPPED [ 60%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_tool_async_def_function SKIPPED [ 60%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_tool_async_def_raises_propagates SKIPPED [ 60%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_submit_with_typed_signature SKIPPED [ 60%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_submit_positional_args SKIPPED [ 60%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_submit_multi_output SKIPPED [ 60%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_submit_wrong_arg_count SKIPPED [ 60%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_extract_parameters SKIPPED [ 60%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_extract_parameters_complex_types SKIPPED [ 60%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_large_variable_injection SKIPPED [ 60%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_large_variable_content_integrity SKIPPED [ 60%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_mixed_small_and_large_variables SKIPPED [ 60%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_multiple_large_variables SKIPPED [ 60%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_large_list_variable SKIPPED [ 61%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_nested_sets_and_tuples SKIPPED [ 61%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_small_variable_not_using_filesystem SKIPPED [ 61%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_large_variable_threshold_boundary SKIPPED [ 61%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_enable_read_paths_symlink SKIPPED [ 61%]
[dspy-pytest] tests/primitives/test_python_interpreter.py::test_enable_read_paths_multiple_files SKIPPED [ 61%]
[dspy-pytest] tests/primitives/test_sandbox_serializable.py::TestABCConformance::test_subclass_conformance PASSED [ 61%]
[dspy-pytest] tests/primitives/test_sandbox_serializable.py::TestABCConformance::test_incomplete_subclass_cannot_instantiate PASSED [ 61%]
[dspy-pytest] tests/primitives/test_sandbox_serializable.py::TestABCConformance::test_structural_conformance_no_longer_accepted PASSED [ 61%]
[dspy-pytest] tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_sandbox_setup PASSED [ 61%]
[dspy-pytest] tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_to_sandbox_returns_bytes PASSED [ 61%]
[dspy-pytest] tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_sandbox_assignment PASSED [ 61%]
[dspy-pytest] tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_rlm_preview PASSED [ 61%]
[dspy-pytest] tests/primitives/test_sandbox_serializable.py::TestCoreMethods::test_rlm_preview_truncation PASSED [ 61%]
[dspy-pytest] tests/primitives/test_sandbox_serializable.py::TestBuildReplVariable::test_builds_variable_with_preview_from_rlm_preview PASSED [ 62%]
[dspy-pytest] tests/primitives/test_sandbox_serializable.py::TestBuildReplVariable::test_surfaces_sandbox_setup_in_description PASSED [ 62%]
[dspy-pytest] tests/primitives/test_sandbox_serializable.py::TestBuildReplVariable::test_passes_field_info_through PASSED [ 62%]
[dspy-pytest] tests/primitives/test_sandbox_serializable.py::TestToReplVariableMethod::test_default_to_repl_variable PASSED [ 62%]
[dspy-pytest] tests/primitives/test_sandbox_serializable.py::TestSignatureAnnotation::test_subclass_supports_signature_annotation PASSED [ 62%]
[dspy-pytest] tests/propose/test_grounded_proposer.py::test_propose_instructions_for_program[None] PASSED [ 62%]
[dspy-pytest] tests/propose/test_grounded_proposer.py::test_propose_instructions_for_program[demo_candidates1] PASSED [ 62%]
[dspy-pytest] tests/propose/test_grounded_proposer.py::test_propose_instruction_for_predictor[None] PASSED [ 62%]
[dspy-pytest] tests/propose/test_grounded_proposer.py::test_propose_instruction_for_predictor[demo_candidates1] PASSED [ 62%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-input_formats-test_markdown_1-input1] SKIPPED [ 62%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-input_formats-test_markdown_1-input2] SKIPPED [ 62%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_many_types_1-input1] SKIPPED [ 62%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_many_types_1-input2] SKIPPED [ 62%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_nesting_2-input1] SKIPPED [ 63%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_nesting_1-input1] SKIPPED [ 63%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-complex_types-test_nesting_1-input2] SKIPPED [ 63%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-input_formats-test_markdown_1-input1] SKIPPED [ 63%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-input_formats-test_markdown_1-input2] SKIPPED [ 63%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_many_types_1-input1] SKIPPED [ 63%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_many_types_1-input2] SKIPPED [ 63%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_nesting_2-input1] SKIPPED [ 63%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_nesting_1-input1] SKIPPED [ 63%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4o-mini-complex_types-test_nesting_1-input2] SKIPPED [ 63%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-input_formats-test_markdown_1-input1] SKIPPED [ 63%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-input_formats-test_markdown_1-input2] SKIPPED [ 63%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_many_types_1-input1] SKIPPED [ 63%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_many_types_1-input2] SKIPPED [ 64%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_nesting_2-input1] SKIPPED [ 64%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_nesting_1-input1] SKIPPED [ 64%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-4-turbo-complex_types-test_nesting_1-input2] SKIPPED [ 64%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-input_formats-test_markdown_1-input1] SKIPPED [ 64%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-input_formats-test_markdown_1-input2] SKIPPED [ 64%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_many_types_1-input1] SKIPPED [ 64%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_many_types_1-input2] SKIPPED [ 64%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_nesting_2-input1] SKIPPED [ 64%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_nesting_1-input1] SKIPPED [ 64%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-preview-complex_types-test_nesting_1-input2] SKIPPED [ 64%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-input_formats-test_markdown_1-input1] SKIPPED [ 64%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-input_formats-test_markdown_1-input2] SKIPPED [ 64%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_many_types_1-input1] SKIPPED [ 65%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_many_types_1-input2] SKIPPED [ 65%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_nesting_2-input1] SKIPPED [ 65%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_nesting_1-input1] SKIPPED [ 65%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gpt-o1-mini-complex_types-test_nesting_1-input2] SKIPPED [ 65%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-input_formats-test_markdown_1-input1] SKIPPED [ 65%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-input_formats-test_markdown_1-input2] SKIPPED [ 65%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_many_types_1-input1] SKIPPED [ 65%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_many_types_1-input2] SKIPPED [ 65%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_nesting_2-input1] SKIPPED [ 65%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_nesting_1-input1] SKIPPED [ 65%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-sonnet-complex_types-test_nesting_1-input2] SKIPPED [ 65%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-input_formats-test_markdown_1-input1] SKIPPED [ 65%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-input_formats-test_markdown_1-input2] SKIPPED [ 66%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_many_types_1-input1] SKIPPED [ 66%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_many_types_1-input2] SKIPPED [ 66%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_nesting_2-input1] SKIPPED [ 66%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_nesting_1-input1] SKIPPED [ 66%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[claude-3.5-haiku-complex_types-test_nesting_1-input2] SKIPPED [ 66%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-input_formats-test_markdown_1-input1] SKIPPED [ 66%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-input_formats-test_markdown_1-input2] SKIPPED [ 66%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_many_types_1-input1] SKIPPED [ 66%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_many_types_1-input2] SKIPPED [ 66%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_nesting_2-input1] SKIPPED [ 66%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_nesting_1-input1] SKIPPED [ 66%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-pro-complex_types-test_nesting_1-input2] SKIPPED [ 66%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-input_formats-test_markdown_1-input1] SKIPPED [ 67%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-input_formats-test_markdown_1-input2] SKIPPED [ 67%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_many_types_1-input1] SKIPPED [ 67%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_many_types_1-input2] SKIPPED [ 67%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_nesting_2-input1] SKIPPED [ 67%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_nesting_1-input1] SKIPPED [ 67%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[gemini-1.5-flash-complex_types-test_nesting_1-input2] SKIPPED [ 67%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 67%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 67%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 67%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 67%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 67%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 67%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-405b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 68%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 68%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 68%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 68%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 68%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 68%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 68%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-70b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 68%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 68%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 68%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 68%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 68%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 68%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 69%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.1-8b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 69%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-input_formats-test_markdown_1-input1] SKIPPED [ 69%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-input_formats-test_markdown_1-input2] SKIPPED [ 69%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_many_types_1-input1] SKIPPED [ 69%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_many_types_1-input2] SKIPPED [ 69%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_nesting_2-input1] SKIPPED [ 69%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_nesting_1-input1] SKIPPED [ 69%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[llama-3.2-3b-instruct-complex_types-test_nesting_1-input2] SKIPPED [ 69%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-input_formats-test_markdown_1-input1] SKIPPED [ 69%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-input_formats-test_markdown_1-input2] SKIPPED [ 69%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_many_types_1-input1] SKIPPED [ 69%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_many_types_1-input2] SKIPPED [ 69%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_nesting_2-input1] SKIPPED [ 69%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_nesting_1-input1] SKIPPED [ 70%]
[dspy-pytest] tests/reliability/test_generated.py::test_generated_cases[deepseek-r1-complex_types-test_nesting_1-input2] SKIPPED [ 70%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-4o] SKIPPED [ 70%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-4o-mini] SKIPPED [ 70%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-4-turbo] SKIPPED [ 70%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-o1-preview] SKIPPED [ 70%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gpt-o1-mini] SKIPPED [ 70%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[claude-3.5-sonnet] SKIPPED [ 70%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[claude-3.5-haiku] SKIPPED [ 70%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gemini-1.5-pro] SKIPPED [ 70%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[gemini-1.5-flash] SKIPPED [ 70%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.1-405b-instruct] SKIPPED [ 70%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.1-70b-instruct] SKIPPED [ 70%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.1-8b-instruct] SKIPPED [ 71%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[llama-3.2-3b-instruct] SKIPPED [ 71%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_qa_with_pydantic_answer_model[deepseek-r1] SKIPPED [ 71%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-Predict] SKIPPED [ 71%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-ChainOfThought] SKIPPED [ 71%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-mini-Predict] SKIPPED [ 71%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4o-mini-ChainOfThought] SKIPPED [ 71%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4-turbo-Predict] SKIPPED [ 71%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-4-turbo-ChainOfThought] SKIPPED [ 71%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-preview-Predict] SKIPPED [ 71%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-preview-ChainOfThought] SKIPPED [ 71%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-mini-Predict] SKIPPED [ 71%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gpt-o1-mini-ChainOfThought] SKIPPED [ 71%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-sonnet-Predict] SKIPPED [ 72%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-sonnet-ChainOfThought] SKIPPED [ 72%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-haiku-Predict] SKIPPED [ 72%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[claude-3.5-haiku-ChainOfThought] SKIPPED [ 72%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-pro-Predict] SKIPPED [ 72%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-pro-ChainOfThought] SKIPPED [ 72%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-flash-Predict] SKIPPED [ 72%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[gemini-1.5-flash-ChainOfThought] SKIPPED [ 72%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-405b-instruct-Predict] SKIPPED [ 72%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-405b-instruct-ChainOfThought] SKIPPED [ 72%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-70b-instruct-Predict] SKIPPED [ 72%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-70b-instruct-ChainOfThought] SKIPPED [ 72%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-8b-instruct-Predict] SKIPPED [ 72%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.1-8b-instruct-ChainOfThought] SKIPPED [ 73%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.2-3b-instruct-Predict] SKIPPED [ 73%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[llama-3.2-3b-instruct-ChainOfThought] SKIPPED [ 73%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[deepseek-r1-Predict] SKIPPED [ 73%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_color_classification_using_enum[deepseek-r1-ChainOfThought] SKIPPED [ 73%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-4o] SKIPPED [ 73%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-4o-mini] SKIPPED [ 73%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-4-turbo] SKIPPED [ 73%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-o1-preview] SKIPPED [ 73%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gpt-o1-mini] SKIPPED [ 73%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[claude-3.5-sonnet] SKIPPED [ 73%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[claude-3.5-haiku] SKIPPED [ 73%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gemini-1.5-pro] SKIPPED [ 73%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[gemini-1.5-flash] SKIPPED [ 74%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.1-405b-instruct] SKIPPED [ 74%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.1-70b-instruct] SKIPPED [ 74%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.1-8b-instruct] SKIPPED [ 74%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[llama-3.2-3b-instruct] SKIPPED [ 74%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_entity_extraction_with_multiple_primitive_outputs[deepseek-r1] SKIPPED [ 74%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-Predict] SKIPPED [ 74%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-ChainOfThought] SKIPPED [ 74%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-mini-Predict] SKIPPED [ 74%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4o-mini-ChainOfThought] SKIPPED [ 74%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4-turbo-Predict] SKIPPED [ 74%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-4-turbo-ChainOfThought] SKIPPED [ 74%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-preview-Predict] SKIPPED [ 74%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-preview-ChainOfThought] SKIPPED [ 75%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-mini-Predict] SKIPPED [ 75%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gpt-o1-mini-ChainOfThought] SKIPPED [ 75%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-sonnet-Predict] SKIPPED [ 75%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-sonnet-ChainOfThought] SKIPPED [ 75%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-haiku-Predict] SKIPPED [ 75%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[claude-3.5-haiku-ChainOfThought] SKIPPED [ 75%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-pro-Predict] SKIPPED [ 75%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-pro-ChainOfThought] SKIPPED [ 75%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-flash-Predict] SKIPPED [ 75%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[gemini-1.5-flash-ChainOfThought] SKIPPED [ 75%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-405b-instruct-Predict] SKIPPED [ 75%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-405b-instruct-ChainOfThought] SKIPPED [ 75%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-70b-instruct-Predict] SKIPPED [ 76%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-70b-instruct-ChainOfThought] SKIPPED [ 76%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-8b-instruct-Predict] SKIPPED [ 76%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.1-8b-instruct-ChainOfThought] SKIPPED [ 76%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.2-3b-instruct-Predict] SKIPPED [ 76%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[llama-3.2-3b-instruct-ChainOfThought] SKIPPED [ 76%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[deepseek-r1-Predict] SKIPPED [ 76%]
[dspy-pytest] tests/reliability/test_pydantic_models.py::test_tool_calling_with_literals[deepseek-r1-ChainOfThought] SKIPPED [ 76%]
[dspy-pytest] tests/retrievers/test_colbertv2.py::test_get_request_raises_on_server_error PASSED [ 76%]
[dspy-pytest] tests/retrievers/test_colbertv2.py::test_post_request_raises_on_server_error PASSED [ 76%]
[dspy-pytest] tests/retrievers/test_colbertv2.py::test_get_request_success PASSED      [ 76%]
[dspy-pytest] tests/retrievers/test_colbertv2.py::test_post_request_success PASSED     [ 76%]
[dspy-pytest] tests/retrievers/test_embeddings.py::test_embeddings_basic_search PASSED [ 76%]
[dspy-pytest] tests/retrievers/test_embeddings.py::test_embeddings_multithreaded_search PASSED [ 76%]
[dspy-pytest] tests/retrievers/test_embeddings.py::test_embeddings_save_load PASSED    [ 77%]
[dspy-pytest] tests/retrievers/test_embeddings.py::test_embeddings_from_saved PASSED   [ 77%]
[dspy-pytest] tests/retrievers/test_embeddings.py::test_embeddings_load_nonexistent_path PASSED [ 77%]
[dspy-pytest] tests/retrievers/test_embeddings.py::test_embeddings_with_scores_basic_search PASSED [ 77%]
[dspy-pytest] tests/retrievers/test_embeddings.py::test_embeddings_with_scores_save_load PASSED [ 77%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_from_local_path PASSED  [ 77%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_from_path_method PASSED [ 77%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_from_path_with_custom_filename PASSED [ 77%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_from_bytes PASSED       [ 77%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_from_bytes_with_filename PASSED [ 77%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_from_file_id PASSED     [ 77%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_from_file_id_with_filename PASSED [ 77%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_from_dict_with_file_data PASSED [ 77%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_from_dict_with_file_id PASSED [ 78%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_format_with_file_data PASSED [ 78%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_format_with_file_id PASSED [ 78%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_repr_with_file_data PASSED [ 78%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_repr_with_file_id PASSED [ 78%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_str PASSED              [ 78%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_encode_file_to_dict_from_path PASSED [ 78%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_encode_file_to_dict_from_bytes PASSED [ 78%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_invalid_file_string PASSED   [ 78%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_invalid_dict PASSED          [ 78%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_in_signature PASSED     [ 78%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_list_in_signature PASSED [ 78%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_optional_file_field PASSED   [ 78%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_save_load_file_signature PASSED [ 79%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_frozen PASSED           [ 79%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_with_all_fields PASSED  [ 79%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_path_not_found PASSED   [ 79%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_custom_mime_type PASSED [ 79%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_from_bytes_custom_mime PASSED [ 79%]
[dspy-pytest] tests/signatures/test_adapter_file.py::test_file_data_uri_in_format PASSED [ 79%]
[dspy-pytest] tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case0] PASSED [ 79%]
[dspy-pytest] tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case1] PASSED [ 79%]
[dspy-pytest] tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case2] PASSED [ 79%]
[dspy-pytest] tests/signatures/test_adapter_image.py::test_basic_image_operations[test_case3] PASSED [ 79%]
[dspy-pytest] tests/signatures/test_adapter_image.py::test_save_load_complex_default_types PASSED [ 79%]
[dspy-pytest] tests/signatures/test_adapter_image.py::test_save_load_complex_types[test_case0] PASSED [ 79%]
[dspy-pytest] tests/signatures/test_adapter_image.py::test_save_load_complex_types[test_case1] PASSED [ 80%]
[dspy-pytest] tests/signatures/test_adapter_image.py::test_save_load_pydantic_model PASSED [ 80%]
[dspy-pytest] tests/signatures/test_adapter_image.py::test_optional_image_field PASSED [ 80%]
[dspy-pytest] tests/signatures/test_adapter_image.py::test_image_repr PASSED           [ 80%]
[dspy-pytest] tests/signatures/test_adapter_image.py::test_from_methods_warn PASSED    [ 80%]
[dspy-pytest] tests/signatures/test_adapter_image.py::test_invalid_string_format PASSED [ 80%]
[dspy-pytest] tests/signatures/test_adapter_image.py::test_pil_image_with_download_parameter PASSED [ 80%]
[dspy-pytest] tests/signatures/test_custom_types.py::test_basic_custom_type_resolution PASSED [ 80%]
[dspy-pytest] tests/signatures/test_custom_types.py::test_type_alias_for_nested_types PASSED [ 80%]
[dspy-pytest] tests/signatures/test_custom_types.py::test_module_level_type_resolution PASSED [ 80%]
[dspy-pytest] tests/signatures/test_custom_types.py::test_recommended_patterns PASSED  [ 80%]
[dspy-pytest] tests/signatures/test_custom_types.py::test_expected_failure PASSED      [ 80%]
[dspy-pytest] tests/signatures/test_custom_types.py::test_module_type_resolution PASSED [ 80%]
[dspy-pytest] tests/signatures/test_signature.py::test_field_types_and_custom_attributes PASSED [ 81%]
[dspy-pytest] tests/signatures/test_signature.py::test_no_input_output PASSED          [ 81%]
[dspy-pytest] tests/signatures/test_signature.py::test_no_input_output2 PASSED         [ 81%]
[dspy-pytest] tests/signatures/test_signature.py::test_all_fields_have_prefix PASSED   [ 81%]
[dspy-pytest] tests/signatures/test_signature.py::test_signature_parsing PASSED        [ 81%]
[dspy-pytest] tests/signatures/test_signature.py::test_duplicate_input_output_field_names_raise PASSED [ 81%]
[dspy-pytest] tests/signatures/test_signature.py::test_with_signature PASSED           [ 81%]
[dspy-pytest] tests/signatures/test_signature.py::test_append_instructions PASSED      [ 81%]
[dspy-pytest] tests/signatures/test_signature.py::test_append_instructions_chains PASSED [ 81%]
[dspy-pytest] tests/signatures/test_signature.py::test_append_instructions_preserves_default_instructions PASSED [ 81%]
[dspy-pytest] tests/signatures/test_signature.py::test_with_updated_field PASSED       [ 81%]
[dspy-pytest] tests/signatures/test_signature.py::test_empty_signature PASSED          [ 81%]
[dspy-pytest] tests/signatures/test_signature.py::test_instructions_signature PASSED   [ 81%]
[dspy-pytest] tests/signatures/test_signature.py::test_signature_instructions PASSED   [ 82%]
[dspy-pytest] tests/signatures/test_signature.py::test_signature_instructions_none PASSED [ 82%]
[dspy-pytest] tests/signatures/test_signature.py::test_signature_from_dict PASSED      [ 82%]
[dspy-pytest] tests/signatures/test_signature.py::test_signature_equality PASSED       [ 82%]
[dspy-pytest] tests/signatures/test_signature.py::test_signature_inequality PASSED     [ 82%]
[dspy-pytest] tests/signatures/test_signature.py::test_equality_format PASSED          [ 82%]
[dspy-pytest] tests/signatures/test_signature.py::test_signature_reverse PASSED        [ 82%]
[dspy-pytest] tests/signatures/test_signature.py::test_insert_field_at_various_positions PASSED [ 82%]
[dspy-pytest] tests/signatures/test_signature.py::test_order_preserved_with_mixed_annotations PASSED [ 82%]
[dspy-pytest] tests/signatures/test_signature.py::test_infer_prefix PASSED             [ 82%]
[dspy-pytest] tests/signatures/test_signature.py::test_insantiating PASSED             [ 82%]
[dspy-pytest] tests/signatures/test_signature.py::test_insantiating2 PASSED            [ 82%]
[dspy-pytest] tests/signatures/test_signature.py::test_multiline_instructions PASSED   [ 82%]
[dspy-pytest] tests/signatures/test_signature.py::test_dump_and_load_state PASSED      [ 83%]
[dspy-pytest] tests/signatures/test_signature.py::test_typed_signatures_basic_types PASSED [ 83%]
[dspy-pytest] tests/signatures/test_signature.py::test_typed_signatures_generics PASSED [ 83%]
[dspy-pytest] tests/signatures/test_signature.py::test_typed_signatures_unions_and_optionals PASSED [ 83%]
[dspy-pytest] tests/signatures/test_signature.py::test_typed_signatures_any PASSED     [ 83%]
[dspy-pytest] tests/signatures/test_signature.py::test_typed_signatures_nested PASSED  [ 83%]
[dspy-pytest] tests/signatures/test_signature.py::test_typed_signatures_from_dict PASSED [ 83%]
[dspy-pytest] tests/signatures/test_signature.py::test_typed_signatures_complex_combinations PASSED [ 83%]
[dspy-pytest] tests/signatures/test_signature.py::test_make_signature_from_string PASSED [ 83%]
[dspy-pytest] tests/signatures/test_signature.py::test_signature_field_with_constraints PASSED [ 83%]
[dspy-pytest] tests/signatures/test_signature.py::test_basic_custom_type PASSED        [ 83%]
[dspy-pytest] tests/signatures/test_signature.py::test_custom_type_from_different_module PASSED [ 83%]
[dspy-pytest] tests/signatures/test_signature.py::test_pep604_union_type_inline PASSED [ 83%]
[dspy-pytest] tests/signatures/test_signature.py::test_pep604_union_type_inline_equivalence PASSED [ 84%]
[dspy-pytest] tests/signatures/test_signature.py::test_pep604_union_type_inline_nested PASSED [ 84%]
[dspy-pytest] tests/signatures/test_signature.py::test_pep604_union_type_class_nested PASSED [ 84%]
[dspy-pytest] tests/signatures/test_signature.py::test_pep604_union_type_class_equivalence PASSED [ 84%]
[dspy-pytest] tests/signatures/test_signature.py::test_pep604_union_type_insert PASSED [ 84%]
[dspy-pytest] tests/signatures/test_signature.py::test_pep604_union_type_with_custom_types PASSED [ 84%]
[dspy-pytest] tests/signatures/test_signature.py::test_signature_cloudpickle_roundtrip PASSED [ 84%]
[dspy-pytest] tests/signatures/test_signature.py::test_predict_cloudpickle_roundtrip PASSED [ 84%]
[dspy-pytest] tests/streaming/test_streaming.py::test_streamify_yields_expected_response_chunks PASSED [ 84%]
[dspy-pytest] tests/streaming/test_streaming.py::test_streaming_response_yields_expected_response_chunks PASSED [ 84%]
[dspy-pytest] tests/streaming/test_streaming.py::test_default_status_streaming PASSED  [ 84%]
[dspy-pytest] tests/streaming/test_streaming.py::test_custom_status_streaming PASSED   [ 84%]
[dspy-pytest] tests/streaming/test_streaming.py::test_concurrent_status_message_providers PASSED [ 84%]
[dspy-pytest] tests/streaming/test_streaming.py::test_stream_listener_chat_adapter SKIPPED [ 84%]
[dspy-pytest] tests/streaming/test_streaming.py::test_default_status_streaming_in_async_program PASSED [ 85%]
[dspy-pytest] tests/streaming/test_streaming.py::test_stream_listener_json_adapter SKIPPED [ 85%]
[dspy-pytest] tests/streaming/test_streaming.py::test_streaming_handles_space_correctly PASSED [ 85%]
[dspy-pytest] tests/streaming/test_streaming.py::test_sync_streaming SKIPPED (need
[dspy-pytest] --llm_call option to run)                                                [ 85%]
[dspy-pytest] tests/streaming/test_streaming.py::test_sync_status_streaming PASSED     [ 85%]
[dspy-pytest] tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_chat_adapter PASSED [ 85%]
[dspy-pytest] tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_json_adapter PASSED [ 85%]
[dspy-pytest] tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_chat_adapter_untokenized_stream PASSED [ 85%]
[dspy-pytest] tests/streaming/test_streaming.py::test_stream_listener_missing_completion_marker_chat_adapter PASSED [ 85%]
[dspy-pytest] tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_json_adapter_untokenized_stream PASSED [ 85%]
[dspy-pytest] tests/streaming/test_streaming.py::test_status_message_non_blocking PASSED [ 85%]
[dspy-pytest] tests/streaming/test_streaming.py::test_status_message_non_blocking_async_program PASSED [ 85%]
[dspy-pytest] tests/streaming/test_streaming.py::test_stream_listener_allow_reuse PASSED [ 85%]
[dspy-pytest] tests/streaming/test_streaming.py::test_stream_listener_returns_correct_chunk_xml_adapter PASSED [ 86%]
[dspy-pytest] tests/streaming/test_streaming.py::test_streaming_allows_custom_chunk_types PASSED [ 86%]
[dspy-pytest] tests/streaming/test_streaming.py::test_streaming_allows_custom_streamable_type PASSED [ 86%]
[dspy-pytest] tests/streaming/test_streaming.py::test_streaming_with_citations PASSED  [ 86%]
[dspy-pytest] tests/streaming/test_streaming.py::test_chat_adapter_simple_pydantic_streaming PASSED [ 86%]
[dspy-pytest] tests/streaming/test_streaming.py::test_chat_adapter_with_generic_type_annotation PASSED [ 86%]
[dspy-pytest] tests/streaming/test_streaming.py::test_chat_adapter_nested_pydantic_streaming PASSED [ 86%]
[dspy-pytest] tests/streaming/test_streaming.py::test_chat_adapter_mixed_fields_streaming PASSED [ 86%]
[dspy-pytest] tests/streaming/test_streaming.py::test_json_adapter_simple_pydantic_streaming PASSED [ 86%]
[dspy-pytest] tests/streaming/test_streaming.py::test_json_adapter_bracket_balance_detection PASSED [ 86%]
[dspy-pytest] tests/streaming/test_streaming.py::test_json_adapter_multiple_fields_detection PASSED [ 86%]
[dspy-pytest] tests/streaming/test_streaming.py::test_stream_listener_could_form_end_identifier_chat_adapter PASSED [ 86%]
[dspy-pytest] tests/streaming/test_streaming.py::test_stream_listener_could_form_end_identifier_json_adapter PASSED [ 86%]
[dspy-pytest] tests/streaming/test_streaming.py::test_stream_listener_could_form_end_identifier_xml_adapter PASSED [ 87%]
[dspy-pytest] tests/streaming/test_streaming.py::test_streaming_reasoning_model PASSED [ 87%]
[dspy-pytest] tests/streaming/test_streaming.py::test_stream_listener_empty_last_chunk_chat_adapter PASSED [ 87%]
[dspy-pytest] tests/streaming/test_streaming.py::test_stream_listener_empty_last_chunk_json_adapter PASSED [ 87%]
[dspy-pytest] tests/streaming/test_streaming.py::test_streaming_reasoning_fallback PASSED [ 87%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_bettertogether_import PASSED [ 87%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_bettertogether_initialization_default PASSED [ 87%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_bettertogether_initialization_custom PASSED [ 87%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_bettertogether_initialization_invalid_optimizer PASSED [ 87%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_strategy_validation PASSED [ 87%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_compile_basic PASSED       [ 87%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_trainset_validation PASSED [ 87%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_valset_ratio_validation PASSED [ 87%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_optimizer_compile_args_validation PASSED [ 88%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_student_in_optimizer_compile_args PASSED [ 88%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_compile_args_passed_to_optimizer PASSED [ 88%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_compile_args_multi_optimizer_strategy PASSED [ 88%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_compile_args_override_global_params PASSED [ 88%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_trainset_shuffling_between_steps PASSED [ 88%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_strategy_execution_order PASSED [ 88%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_lm_lifecycle_management PASSED [ 88%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_error_handling_returns_best_program PASSED [ 88%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_program_selection[test_valset0-p_optimized-With valset: returns best score (p), not latest (w)] PASSED [ 88%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_program_selection[None-w_optimized-Without valset: returns latest program (w)] PASSED [ 88%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_candidate_programs_structure PASSED [ 88%]
[dspy-pytest] tests/teleprompt/test_bettertogether.py::test_empty_valset_handling PASSED [ 88%]
[dspy-pytest] tests/teleprompt/test_bootstrap.py::test_bootstrap_initialization PASSED [ 89%]
[dspy-pytest] tests/teleprompt/test_bootstrap.py::test_compile_with_predict_instances PASSED [ 89%]
[dspy-pytest] tests/teleprompt/test_bootstrap.py::test_bootstrap_effectiveness PASSED  [ 89%]
[dspy-pytest] tests/teleprompt/test_bootstrap.py::test_error_handling_during_bootstrap PASSED [ 89%]
[dspy-pytest] tests/teleprompt/test_bootstrap.py::test_validation_set_usage PASSED     [ 89%]
[dspy-pytest] tests/teleprompt/test_bootstrap_finetune.py::test_bootstrap_finetune_initialization PASSED [ 89%]
[dspy-pytest] tests/teleprompt/test_bootstrap_finetune.py::test_compile_with_predict_instances PASSED [ 89%]
[dspy-pytest] tests/teleprompt/test_bootstrap_finetune.py::test_error_handling_missing_lm PASSED [ 89%]
[dspy-pytest] tests/teleprompt/test_bootstrap_trace.py::test_bootstrap_trace_data PASSED [ 89%]
[dspy-pytest] tests/teleprompt/test_bootstrap_trace.py::test_bootstrap_trace_data_passes_callback_metadata PASSED [ 89%]
[dspy-pytest] tests/teleprompt/test_copro_optimizer.py::test_signature_optimizer_initialization PASSED [ 89%]
[dspy-pytest] tests/teleprompt/test_copro_optimizer.py::test_signature_optimizer_optimization_process PASSED [ 89%]
[dspy-pytest] tests/teleprompt/test_copro_optimizer.py::test_signature_optimizer_statistics_tracking PASSED [ 89%]
[dspy-pytest] tests/teleprompt/test_copro_optimizer.py::test_optimization_and_output_verification PASSED [ 90%]
[dspy-pytest] tests/teleprompt/test_copro_optimizer.py::test_statistics_tracking_during_optimization PASSED [ 90%]
[dspy-pytest] tests/teleprompt/test_ensemble.py::test_ensemble_without_reduction PASSED [ 90%]
[dspy-pytest] tests/teleprompt/test_ensemble.py::test_ensemble_with_reduction PASSED   [ 90%]
[dspy-pytest] tests/teleprompt/test_ensemble.py::test_ensemble_with_size_limitation PASSED [ 90%]
[dspy-pytest] tests/teleprompt/test_ensemble.py::test_ensemble_deterministic_behavior PASSED [ 90%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[None-batch0-expected_callback_metadata0] PASSED [ 90%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[None-batch1-expected_callback_metadata1] PASSED [ 90%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[1-batch2-expected_callback_metadata2] PASSED [ 90%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_gepa_adapter_disables_logging_on_minibatch_eval[1-batch3-expected_callback_metadata3] PASSED [ 90%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_basic_workflow[True] PASSED          [ 90%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_basic_workflow[False] PASSED         [ 90%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_workflow_with_custom_instruction_proposer_and_component_selector PASSED [ 90%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_metric_requires_feedback_signature PASSED [ 91%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_reflection_prompt_template_in_gepa_kwargs_raises PASSED [ 91%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_gepa_compile_with_track_usage_no_tuple_error PASSED [ 91%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_component_selector_functionality PASSED [ 91%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_component_selector_default_behavior PASSED [ 91%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_component_selector_string_round_robin PASSED [ 91%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_component_selector_string_all PASSED [ 91%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_component_selector_custom_random PASSED [ 91%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_alternating_half_component_selector PASSED [ 91%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_track_stats_result_structure PASSED  [ 91%]
[dspy-pytest] tests/teleprompt/test_gepa.py::test_track_best_outputs_result_structure PASSED [ 91%]
[dspy-pytest] tests/teleprompt/test_gepa_instruction_proposer.py::test_reflection_lm_gets_structured_images PASSED [ 91%]
[dspy-pytest] tests/teleprompt/test_gepa_instruction_proposer.py::test_custom_proposer_without_reflection_lm PASSED [ 91%]
[dspy-pytest] tests/teleprompt/test_gepa_instruction_proposer.py::test_image_serialization_into_strings PASSED [ 92%]
[dspy-pytest] tests/teleprompt/test_gepa_instruction_proposer.py::test_default_proposer[True] PASSED [ 92%]
[dspy-pytest] tests/teleprompt/test_gepa_instruction_proposer.py::test_default_proposer[False] PASSED [ 92%]
[dspy-pytest] tests/teleprompt/test_grpo.py::test_grpo_dataset_shuffler PASSED         [ 92%]
[dspy-pytest] tests/teleprompt/test_grpo.py::test_grpo_dataset_shuffler_with_num_ex_per_step_less_dataset PASSED [ 92%]
[dspy-pytest] tests/teleprompt/test_grpo.py::test_grpo_dataset_shuffler_with_num_ex_per_step_greater_dataset PASSED [ 92%]
[dspy-pytest] tests/teleprompt/test_knn_fewshot.py::test_knn_few_shot_initialization PASSED [ 92%]
[dspy-pytest] tests/teleprompt/test_random_search.py::test_basic_workflow PASSED       [ 92%]
[dspy-pytest] tests/teleprompt/test_teleprompt.py::test_get_params PASSED              [ 92%]
[dspy-pytest] tests/teleprompt/test_utils.py::test_eval_candidate_program_full_trainset PASSED [ 92%]
[dspy-pytest] tests/teleprompt/test_utils.py::test_eval_candidate_program_minibatch PASSED [ 92%]
[dspy-pytest] tests/teleprompt/test_utils.py::test_eval_candidate_program_failure PASSED [ 92%]
[dspy-pytest] tests/teleprompt/test_utils.py::test_create_n_fewshot_demo_sets_passes_metric_threshold_for_unshuffled PASSED [ 92%]
[dspy-pytest] tests/utils/test_annotation.py::test_experimental_decorator_on_function PASSED [ 92%]
[dspy-pytest] tests/utils/test_annotation.py::test_experimental_decorator_on_function_with_version PASSED [ 93%]
[dspy-pytest] tests/utils/test_annotation.py::test_experimental_decorator_on_class PASSED [ 93%]
[dspy-pytest] tests/utils/test_annotation.py::test_experimental_decorator_on_class_with_version PASSED [ 93%]
[dspy-pytest] tests/utils/test_annotation.py::test_experimental_decorator_without_docstring PASSED [ 93%]
[dspy-pytest] tests/utils/test_annotation.py::test_experimental_decorator_without_docstring_with_version PASSED [ 93%]
[dspy-pytest] tests/utils/test_annotation.py::test_experimental_decorator_with_callable_syntax PASSED [ 93%]
[dspy-pytest] tests/utils/test_annotation.py::test_experimental_decorator_with_version_callable_syntax PASSED [ 93%]
[dspy-pytest] tests/utils/test_asyncify.py::test_async_limiter PASSED                  [ 93%]
[dspy-pytest] tests/utils/test_asyncify.py::test_asyncify PASSED                       [ 93%]
[dspy-pytest] tests/utils/test_exceptions.py::test_lm_errors_are_exported_from_dspy PASSED [ 93%]
[dspy-pytest] tests/utils/test_exceptions.py::test_retryable_lm_errors_classification PASSED [ 93%]
[dspy-pytest] tests/utils/test_exceptions.py::test_lm_error_metadata PASSED            [ 93%]
[dspy-pytest] tests/utils/test_exceptions.py::test_context_window_exceeded_error_defaults PASSED [ 93%]
[dspy-pytest] tests/utils/test_exceptions.py::test_context_window_exceeded_error_with_model PASSED [ 94%]
[dspy-pytest] tests/utils/test_exceptions.py::test_context_window_exceeded_error_with_message PASSED [ 94%]
[dspy-pytest] tests/utils/test_exceptions.py::test_context_window_exceeded_error_message_without_model PASSED [ 94%]
[dspy-pytest] tests/utils/test_exceptions.py::test_adapter_parse_error_basic PASSED    [ 94%]
[dspy-pytest] tests/utils/test_exceptions.py::test_adapter_parse_error_with_message PASSED [ 94%]
[dspy-pytest] tests/utils/test_exceptions.py::test_adapter_parse_error_with_parsed_result PASSED [ 94%]
[dspy-pytest] tests/utils/test_lazy_import.py::test_is_available_true_for_stdlib PASSED [ 94%]
[dspy-pytest] tests/utils/test_lazy_import.py::test_is_available_false_for_missing PASSED [ 94%]
[dspy-pytest] tests/utils/test_lazy_import.py::test_is_available_does_not_import_module PASSED [ 94%]
[dspy-pytest] tests/utils/test_lazy_import.py::test_require_returns_lazy_module_when_present PASSED [ 94%]
[dspy-pytest] tests/utils/test_lazy_import.py::test_require_returns_cached_module PASSED [ 94%]
[dspy-pytest] tests/utils/test_lazy_import.py::test_require_is_safe_under_concurrent_first_use PASSED [ 94%]
[dspy-pytest] tests/utils/test_lazy_import.py::test_require_assignment_updates_materialized_module PASSED [ 94%]
[dspy-pytest] tests/utils/test_lazy_import.py::test_require_returns_stub_when_missing PASSED [ 95%]
[dspy-pytest] tests/utils/test_lazy_import.py::test_require_stub_raises_on_access_with_install_hint PASSED [ 95%]
[dspy-pytest] tests/utils/test_lazy_import.py::test_require_stub_uses_install_hint_for_litellm PASSED [ 95%]
[dspy-pytest] tests/utils/test_lazy_import.py::test_require_stub_uses_explicit_extra PASSED [ 95%]
[dspy-pytest] tests/utils/test_lazy_import.py::test_require_stub_falls_back_to_module_name PASSED [ 95%]
[dspy-pytest] tests/utils/test_lazy_import.py::test_install_hints_match_pyproject_extras PASSED [ 95%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[l-value0] PASSED       [ 95%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[t.t.t.t.z-z] PASSED    [ 95%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[a[0]-0] PASSED         [ 95%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[a[1][0]-1] PASSED      [ 95%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[a[1][2]-value4] PASSED [ 95%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[b["x"]-value5] PASSED  [ 95%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[b["x"]["y"]-y] PASSED  [ 95%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[b["z"]-value7] PASSED  [ 96%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[b["z"][1]-2] PASSED    [ 96%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[b["w"].z-z] PASSED     [ 96%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[b["w"].t.l-value10] PASSED [ 96%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z] PASSED      [ 96%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[l[-1]-2] PASSED        [ 96%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.a[-1].z-z] PASSED [ 96%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.b["z"][0]-1] PASSED [ 96%]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[a[-1].t.z-z] PASSED    [ 96%]
[dspy-pytest] tests/utils/test_magicattr.py::test_person_example PASSED                [ 96%]
[dspy-pytest] tests/utils/test_magicattr.py::test_empty PASSED                         [ 96%]
[dspy-pytest] tests/utils/test_mcp.py::test_convert_mcp_tool SKIPPED (need --extra
[dspy-pytest] option to run)                                                           [ 96%]
[dspy-pytest] tests/utils/test_parallelizer.py::test_worker_threads_independence PASSED [ 96%]
[dspy-pytest] tests/utils/test_parallelizer.py::test_parallel_execution_speed PASSED   [ 97%]
[dspy-pytest] tests/utils/test_parallelizer.py::test_max_errors_handling PASSED        [ 97%]
[dspy-pytest] tests/utils/test_parallelizer.py::test_max_errors_not_met PASSED         [ 97%]
[dspy-pytest] tests/utils/test_parallelizer.py::test_parallel_executor_tracks_failed_indices_and_exceptions PASSED [ 97%]
[dspy-pytest] tests/utils/test_parallelizer.py::test_sequential_execution_runs_on_main_thread PASSED [ 97%]
[dspy-pytest] tests/utils/test_parallelizer.py::test_sequential_max_errors_not_met PASSED [ 97%]
[dspy-pytest] tests/utils/test_parallelizer.py::test_sequential_max_errors_exceeded PASSED [ 97%]
[dspy-pytest] tests/utils/test_parallelizer.py::test_sequential_tracks_failed_indices_and_exceptions PASSED [ 97%]
[dspy-pytest] tests/utils/test_parallelizer.py::test_sequential_compare_results PASSED [ 97%]
[dspy-pytest] tests/utils/test_saving.py::test_save_predict PASSED                     [ 97%]
[dspy-pytest] tests/utils/test_saving.py::test_save_custom_model PASSED                [ 97%]
[dspy-pytest] tests/utils/test_saving.py::test_save_model_with_custom_signature PASSED [ 97%]
[dspy-pytest] tests/utils/test_saving.py::test_save_compiled_model SKIPPED (need
[dspy-pytest] --extra option to run)                                                   [ 97%]
[dspy-pytest] tests/utils/test_saving.py::test_load_with_version_mismatch PASSED       [ 98%]
[dspy-pytest] tests/utils/test_saving.py::test_pickle_loading_requires_explicit_permission PASSED [ 98%]
[dspy-pytest] tests/utils/test_saving.py::test_pkl_file_loading_requires_explicit_permission PASSED [ 98%]
[dspy-pytest] tests/utils/test_saving.py::test_json_file_loading_works_without_permission PASSED [ 98%]
[dspy-pytest] tests/utils/test_settings.py::test_basic_dspy_settings PASSED            [ 98%]
[dspy-pytest] tests/utils/test_settings.py::test_forbid_configure_call_in_child_thread PASSED [ 98%]
[dspy-pytest] tests/utils/test_settings.py::test_dspy_context PASSED                   [ 98%]
[dspy-pytest] tests/utils/test_settings.py::test_dspy_context_parallel PASSED          [ 98%]
[dspy-pytest] tests/utils/test_settings.py::test_dspy_context_with_dspy_parallel PASSED [ 98%]
[dspy-pytest] tests/utils/test_settings.py::test_dspy_context_with_async_task_group PASSED [ 98%]
[dspy-pytest] tests/utils/test_settings.py::test_dspy_configure_allowance_async PASSED [ 98%]
[dspy-pytest] tests/utils/test_settings.py::test_dspy_settings_save_load PASSED        [ 98%]
[dspy-pytest] tests/utils/test_settings.py::test_dspy_settings_save_exclude_keys PASSED [ 98%]
[dspy-pytest] tests/utils/test_settings.py::test_settings_save_with_extra_modules PASSED [ 99%]
[dspy-pytest] tests/utils/test_syncify.py::test_syncify_in_place PASSED                [ 99%]
[dspy-pytest] tests/utils/test_syncify.py::test_syncify_with_wrapper PASSED            [ 99%]
[dspy-pytest] tests/utils/test_syncify.py::test_syncify_works_with_optimizers PASSED   [ 99%]
[dspy-pytest] tests/utils/test_unbatchify.py::test_unbatchify_batch_size_trigger PASSED [ 99%]
[dspy-pytest] tests/utils/test_unbatchify.py::test_unbatchify_timeout_trigger PASSED   [ 99%]
[dspy-pytest] tests/utils/test_usage_tracker.py::test_add_usage_entry PASSED           [ 99%]
[dspy-pytest] tests/utils/test_usage_tracker.py::test_get_total_tokens PASSED          [ 99%]
[dspy-pytest] tests/utils/test_usage_tracker.py::test_track_usage_with_multiple_models PASSED [ 99%]
[dspy-pytest] tests/utils/test_usage_tracker.py::test_track_usage_context_manager SKIPPED [ 99%]
[dspy-pytest] tests/utils/test_usage_tracker.py::test_merge_usage_entries_with_new_keys PASSED [ 99%]
[dspy-pytest] tests/utils/test_usage_tracker.py::test_merge_usage_entries_with_none_values PASSED [ 99%]
[dspy-pytest] tests/utils/test_usage_tracker.py::test_merge_usage_entries_with_pydantic_models PASSED [ 99%]
[dspy-pytest] tests/utils/test_usage_tracker.py::test_parallel_executor_with_usage_tracker PASSED [100%]
[dspy-pytest] 
[dspy-pytest] =============================== warnings summary ===============================
[dspy-pytest] tests/clients/test_lm.py::test_chat_lms_can_be_queried
[dspy-pytest] tests/clients/test_lm.py::test_lm_calls_support_callables
[dspy-pytest] tests/clients/test_lm.py::test_lm_calls_support_pydantic_models
[dspy-pytest]   /nix/store/rn0r480kscbk8159qi3qjflil71kqglq-dspy-test-env/lib/python3.13/site-packages/pydantic/main.py:464: UserWarning: Pydantic serializer warnings:
[dspy-pytest]     PydanticSerializationUnexpectedValue(Expected 9 fields but got 5: Expected `Message` - serialized value may not be as expected [field_name='choices', input_value=Message(content='Hi!', ro...ields={'refusal': None}), input_type=Message])
[dspy-pytest]     PydanticSerializationUnexpectedValue(Expected `StreamingChoices` - serialized value may not be as expected [field_name='choices', input_value=Choices(finish_reason='st...elds={'refusal': None})), input_type=Choices])
[dspy-pytest]     return self.__pydantic_serializer__.to_python(
[dspy-pytest] 
[dspy-pytest] tests/predict/test_multi_chain_comparison.py::test_basic_example
[dspy-pytest] tests/predict/test_multi_chain_comparison.py::test_basic_example
[dspy-pytest] tests/predict/test_multi_chain_comparison.py::test_basic_example
[dspy-pytest]   /build/r9a1svd8gqmcs0gcy8lz2ssr6798r95l-source/dspy/predict/multi_chain_comparison.py:19: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version.
[dspy-pytest]     InputField(
[dspy-pytest] 
[dspy-pytest] tests/predict/test_multi_chain_comparison.py::test_basic_example
[dspy-pytest]   /build/r9a1svd8gqmcs0gcy8lz2ssr6798r95l-source/dspy/predict/multi_chain_comparison.py:27: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version.
[dspy-pytest]     OutputField(
[dspy-pytest] 
[dspy-pytest] tests/predict/test_predict.py::test_signature_fields_after_dump_and_load_state
[dspy-pytest]   /build/r9a1svd8gqmcs0gcy8lz2ssr6798r95l-source/tests/predict/test_predict.py:299: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version.
[dspy-pytest]     sentiment = dspy.OutputField(desc="I am a malicious output!", prefix="I am a prefix!")
[dspy-pytest] 
[dspy-pytest] tests/signatures/test_signature.py::test_all_fields_have_prefix
[dspy-pytest]   /build/r9a1svd8gqmcs0gcy8lz2ssr6798r95l-source/tests/signatures/test_signature.py:46: DeprecationWarning: The 'prefix' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version.
[dspy-pytest]     input = InputField(prefix="Modified:")
[dspy-pytest] 
[dspy-pytest] tests/signatures/test_signature.py::test_equality_format
[dspy-pytest]   /build/r9a1svd8gqmcs0gcy8lz2ssr6798r95l-source/tests/signatures/test_signature.py:157: DeprecationWarning: The 'format' argument in InputField/OutputField is deprecated and has no effect in DSPy. It will be removed in a future version.
[dspy-pytest]     input = InputField(format=lambda x: x)
[dspy-pytest] 
[dspy-pytest] tests/streaming/test_streaming.py::test_streamify_yields_expected_response_chunks
[dspy-pytest] tests/streaming/test_streaming.py::test_streaming_response_yields_expected_response_chunks
[dspy-pytest]   /nix/store/rn0r480kscbk8159qi3qjflil71kqglq-dspy-test-env/lib/python3.13/site-packages/pydantic/main.py:464: UserWarning: Pydantic serializer warnings:
[dspy-pytest]     PydanticSerializationUnexpectedValue(Expected 9 fields but got 5: Expected `Message` - serialized value may not be as expected [field_name='choices', input_value=Message(content='{"output...er_specific_fields=None), input_type=Message])
[dspy-pytest]     PydanticSerializationUnexpectedValue(Expected `StreamingChoices` - serialized value may not be as expected [field_name='choices', input_value=Choices(finish_reason='st...r_specific_fields=None)), input_type=Choices])
[dspy-pytest]     return self.__pydantic_serializer__.to_python(
[dspy-pytest] 
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[l[-1]-2]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.a[-1].z-z]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[a[-1].t.z-z]
[dspy-pytest] tests/utils/test_magicattr.py::test_person_example
[dspy-pytest] tests/utils/test_magicattr.py::test_person_example
[dspy-pytest] tests/utils/test_magicattr.py::test_person_example
[dspy-pytest] tests/utils/test_magicattr.py::test_person_example
[dspy-pytest]   /build/r9a1svd8gqmcs0gcy8lz2ssr6798r95l-source/dspy/utils/magicattr.py:99: DeprecationWarning: ast.Num is deprecated and will be removed in Python 3.14; use ast.Constant instead
[dspy-pytest]     if hasattr(ast, "Num") and isinstance(node, ast.Num):
[dspy-pytest] 
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[a[-1].z-z]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[l[-1]-2]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[a[2].t.a[-1].z-z]
[dspy-pytest] tests/utils/test_magicattr.py::test_magicattr_get[a[-1].t.z-z]
[dspy-pytest] tests/utils/test_magicattr.py::test_person_example
[dspy-pytest] tests/utils/test_magicattr.py::test_person_example
[dspy-pytest] tests/utils/test_magicattr.py::test_person_example
[dspy-pytest] tests/utils/test_magicattr.py::test_person_example
[dspy-pytest]   /build/r9a1svd8gqmcs0gcy8lz2ssr6798r95l-source/dspy/utils/magicattr.py:102: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
[dspy-pytest]     elif hasattr(ast, "Str") and isinstance(node, ast.Str):
[dspy-pytest] 
[dspy-pytest] tests/utils/test_settings.py::test_dspy_configure_allowance_async
[dspy-pytest]   /build/r9a1svd8gqmcs0gcy8lz2ssr6798r95l-source/tests/utils/test_settings.py:194: RuntimeWarning: coroutine 'test_dspy_configure_allowance_async.<locals>.foo4' was never awaited
[dspy-pytest]     foo4()
[dspy-pytest]   Enable tracemalloc to get traceback where the object was allocated.
[dspy-pytest]   See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
[dspy-pytest] 
[dspy-pytest] -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
[dspy-pytest] === 984 passed, 330 skipped, 9 deselected, 31 warnings in 1956.54s (0:32:36) ===
[dspy-pytest] checkPhase completed in 32 minutes 40 seconds
[dspy-pytest] Phase: installPhase
[dspy-pytest] Phase: fixupPhase
[dspy-pytest] shrinking RPATHs of ELF executables and libraries in /nix/store/fk3rqwc1r9f58lrw42cwx7qbgxbs3c8k-dspy-pytest
[dspy-pytest] checking for references to /build/ in /nix/store/fk3rqwc1r9f58lrw42cwx7qbgxbs3c8k-dspy-pytest...
[dspy-pytest] patching script interpreter paths in /nix/store/fk3rqwc1r9f58lrw42cwx7qbgxbs3c8k-dspy-pytest
[dspy-pytest:post-build] Uploading to the NixCI staging cache: /nix/store/fk3rqwc1r9f58lrw42cwx7qbgxbs3c8k-dspy-pytest
[dspy-pytest:post-build] warning: 'warn-short-path-literals' is deprecated, use 'lint-short-path-literals = ignore' instead
[dspy-pytest:post-build] copying 1 paths...
[dspy-pytest:post-build] copying path '/nix/store/fk3rqwc1r9f58lrw42cwx7qbgxbs3c8k-dspy-pytest' to 'https://cache.staging.nix-ci.com'...
[dspy-pytest:post-build] warning: 'warn-short-path-literals' is deprecated, use 'lint-short-path-literals = ignore' instead
[dspy-pytest:post-build] copying 1 paths...
[dspy-pytest:post-build] copying path '/nix/store/rcq49d990xqb45ygc83sng306xk2cspr-dspy-pytest.drv' to 'https://cache.staging.nix-ci.com'...
Progress: 1 of 2 built
Progress: 2 of 2 built
/nix/store/fk3rqwc1r9f58lrw42cwx7qbgxbs3c8k-dspy-pytest
Build succeeded.