Skip to content

User encoding is skipped for elements iterated from non-generic .NET containers/enumerators #2679

@brandon-avantus

Description

@brandon-avantus

Environment

  • Pythonnet version: 3.0.5
  • Python version: 3.12.9
  • Operating System: Linux (6.12.63-1-MANJARO # 1 SMP PREEMPT_DYNAMIC Thu, 18 Dec 2025 13:46:28 +0000 x86_64 GNU/Linux)
  • .NET Runtime: coreclr 10.0.0

Details

After creating and registering an encoder (subclass of IPyObjectEncoder) to convert CLR objects to Python equivalents, iterating over non-generic containers yields CLR proxy objects instead of the converted Python object. However, doing the same for generic containers yields the converted objects as expected. In either case, accessing elements using the indexer produces a converted object.

Here is a minimal test case demonstrating the issue:

import pytest

import System
from Python.Test import CodecResetter
from Python.Runtime import IPyObjectEncoder, PyObjectConversions

@pytest.mark.parametrize('container_type', ['generic', 'non-generic'])
def test_iterator_element_conversion(container_type):
    """Test iterator element conversion from Python."""
    from Python.Test import Spam

    try:
        from Python.Test import IteratorElementEncoder
    except ImportError:
        class IteratorElementEncoder(IPyObjectEncoder):
            __namespace__ = "Python.Test"
            def CanEncode(self, clr_type):
                return clr_type.Name == "Spam" and clr_type.Namespace == "Python.Test"
            def TryEncode(self, clr_object):
                return clr_object.GetValue()

    spam_encoder = IteratorElementEncoder()
    PyObjectConversions.RegisterEncoder(spam_encoder)

    values = ["first", "second", "third"]
    if container_type == 'generic':
        container = System.Collections.Generic.List[Spam]()
        for value in values:
            container.Add(Spam(value))
    else:
        container = System.Array[Spam](Spam(v) for v in values)

    assert type(container[0]) is str
    assert next(iter(container.GetEnumerator())) == container[0]
    assert list(container.GetEnumerator()) == values

    CodecResetter.Reset()

And here are the results of running the test:

================================================================= FAILURES =================================================================
______________________________________________ test_iterator_element_conversion[non-generic] _______________________________________________

...
        assert type(container[0]) is str
>       assert next(iter(container.GetEnumerator())) == container[0]
E       AssertionError: assert <Python.Test.Spam object at 0x7ee403c447c0> == 'first'
E        +  where <Python.Test.Spam object at 0x7ee403c447c0> = next(<CLR.Iterator object at 0x7ee403c449c0>)
E        +    where <CLR.Iterator object at 0x7ee403c449c0> = iter(<System.Collections.IEnumerator object at 0x7ee403c46600>)
E        +      where <System.Collections.IEnumerator object at 0x7ee403c46600> = <bound method 'GetEnumerator'>()
E        +        where <bound method 'GetEnumerator'> = <Python.Test.Spam[] object at 0x7f25379bb540>.GetEnumerator

tests/test_conversion.py:786: AssertionError
========================================================= short test summary info ==========================================================
FAILED tests/test_conversion.py::test_iterator_element_conversion[non-generic] - AssertionError: assert <Python.Test.Spam object at 0x7ee403c447c0> == 'first'
======================================================= 1 failed, 1 passed =======================================================

The first assert passes for each test because the indexers of both generic and non-generic containers properly convert the object. The final two asserts both fail for the non-generic version because conversion is not performed, and the Spam proxies are returned instead of being converted to strings.

Implementation Details

The issue stems from src/runtime/Types/ClassBase.cs:ClassBase.tp_iter_impl(), which determines the generic type of a container, defaulting to System.Object for non-generic containers. The element type is then passed to src/runtime/Types/Iterator.cs:Iterator where it is used in calls to Converter.ToPython(). The problem is that System.Object is passed as the element type even though the type is actually something different that could potentially be converted by a user encoder. But System.Object types are not passed on to user converters and are returned as CLR proxies instead.

The solution is to check if the element type is System.Object, and query and use the actual value type instead.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions