Exception: string index out of range when fetching site lists with expand:fields

Describe the bug

I am trying to fetch the list of lists for a given site, with an expand on the fields.
This crashes during deserialisation with a "string index out of range".
The full stack trace is the following

File /usr/local/lib/python3.10/site-packages/msgraph/generated/sites/item/lists/item/items/items_request_builder.py:81, in ItemsRequestBuilder.get(self, request_configuration)
     78     raise Exception("Http core is null") 
     79 from ......models.list_item_collection_response import ListItemCollectionResponse
---> 81 return await self.request_adapter.send_async(request_info, ListItemCollectionResponse, error_mapping)

File /usr/local/lib/python3.10/site-packages/kiota_http/httpx_request_adapter.py:196, in HttpxRequestAdapter.send_async(self, request_info, parsable_factory, error_map)
    194     return None
    195 _deserialized_span = self._start_local_tracing_span("get_object_value", parent_span)
--> 196 value = root_node.get_object_value(parsable_factory)
    197 parent_span.set_attribute(DESERIALIZED_MODEL_NAME_KEY, value.__class__.__name__)
    198 _deserialized_span.end()

File /usr/local/lib/python3.10/site-packages/kiota_serialization_json/json_parse_node.py:222, in JsonParseNode.get_object_value(self, factory)
    220 if on_before := self.on_before_assign_field_values:
    221     on_before(result)
--> 222 self._assign_field_values(result)
    223 if on_after := self.on_after_assign_field_values:
    224     on_after(result)

File /usr/local/lib/python3.10/site-packages/kiota_serialization_json/json_parse_node.py:291, in JsonParseNode._assign_field_values(self, item)
    289         continue
    290     field_deserializer = field_deserializers[field_name]
--> 291     field_deserializer(JsonParseNode(field_value))
    292 elif item_additional_data is not None:
    293     item_additional_data[field_name] = self.try_get_anything(field_value)

File /usr/local/lib/python3.10/site-packages/msgraph/generated/models/list_item_collection_response.py:40, in ListItemCollectionResponse.get_field_deserializers.<locals>.<lambda>(n)
     36 from .base_collection_pagination_count_response import BaseCollectionPaginationCountResponse
     37 from .list_item import ListItem
     39 fields: Dict[str, Callable[[Any], None]] = {
---> 40     "value": lambda n : setattr(self, 'value', n.get_collection_of_object_values(ListItem)),
     41 }
     42 super_fields = super().get_field_deserializers()
     43 fields.update(super_fields)

File /usr/local/lib/python3.10/site-packages/kiota_serialization_json/json_parse_node.py:170, in JsonParseNode.get_collection_of_object_values(self, factory)
    165 """Gets the collection of type U values from the json node
    166 Returns:
    167     List[U]: The collection of model object values of the node
    168 """
    169 if isinstance(self._json_node, list):
--> 170     return list(
    171         map(
    172             lambda x: self._create_new_node(x).get_object_value(factory),  # type: ignore
    173             self._json_node,
    174         )
    175     )
    176 return []

File /usr/local/lib/python3.10/site-packages/kiota_serialization_json/json_parse_node.py:172, in JsonParseNode.get_collection_of_object_values.<locals>.<lambda>(x)
    165 """Gets the collection of type U values from the json node
    166 Returns:
    167     List[U]: The collection of model object values of the node
    168 """
    169 if isinstance(self._json_node, list):
    170     return list(
    171         map(
--> 172             lambda x: self._create_new_node(x).get_object_value(factory),  # type: ignore
    173             self._json_node,
    174         )
    175     )
    176 return []

File /usr/local/lib/python3.10/site-packages/kiota_serialization_json/json_parse_node.py:222, in JsonParseNode.get_object_value(self, factory)
    220 if on_before := self.on_before_assign_field_values:
    221     on_before(result)
--> 222 self._assign_field_values(result)
    223 if on_after := self.on_after_assign_field_values:
    224     on_after(result)

File /usr/local/lib/python3.10/site-packages/kiota_serialization_json/json_parse_node.py:291, in JsonParseNode._assign_field_values(self, item)
    289         continue
    290     field_deserializer = field_deserializers[field_name]
--> 291     field_deserializer(JsonParseNode(field_value))
    292 elif item_additional_data is not None:
    293     item_additional_data[field_name] = self.try_get_anything(field_value)

File /usr/local/lib/python3.10/site-packages/msgraph/generated/models/list_item.py:76, in ListItem.get_field_deserializers.<locals>.<lambda>(n)
     68 from .list_item_version import ListItemVersion
     69 from .sharepoint_ids import SharepointIds
     71 fields: Dict[str, Callable[[Any], None]] = {
     72     "analytics": lambda n : setattr(self, 'analytics', n.get_object_value(ItemAnalytics)),
     73     "contentType": lambda n : setattr(self, 'content_type', n.get_object_value(ContentTypeInfo)),
     74     "documentSetVersions": lambda n : setattr(self, 'document_set_versions', n.get_collection_of_object_values(DocumentSetVersion)),
     75     "driveItem": lambda n : setattr(self, 'drive_item', n.get_object_value(DriveItem)),
---> 76     "fields": lambda n : setattr(self, 'fields', n.get_object_value(FieldValueSet)),
     77     "sharepointIds": lambda n : setattr(self, 'sharepoint_ids', n.get_object_value(SharepointIds)),
     78     "versions": lambda n : setattr(self, 'versions', n.get_collection_of_object_values(ListItemVersion)),
     79 }
     80 super_fields = super().get_field_deserializers()
     81 fields.update(super_fields)

File /usr/local/lib/python3.10/site-packages/kiota_serialization_json/json_parse_node.py:222, in JsonParseNode.get_object_value(self, factory)
    220 if on_before := self.on_before_assign_field_values:
    221     on_before(result)
--> 222 self._assign_field_values(result)
    223 if on_after := self.on_after_assign_field_values:
    224     on_after(result)

File /usr/local/lib/python3.10/site-packages/kiota_serialization_json/json_parse_node.py:293, in JsonParseNode._assign_field_values(self, item)
    291     field_deserializer(JsonParseNode(field_value))
    292 elif item_additional_data is not None:
--> 293     item_additional_data[field_name] = self.try_get_anything(field_value)
    294 else:
    295     warnings.warn(
    296         f"Found additional property {field_name} to \
    297         deserialize but the model doesn't support additional data"
    298     )

File /usr/local/lib/python3.10/site-packages/kiota_serialization_json/json_parse_node.py:316, in JsonParseNode.try_get_anything(self, value)
    313 if self.__is_four_digit_number(value):
    314     return value
--> 316 datetime_obj = pendulum.parse(value)
    317 if isinstance(datetime_obj, pendulum.Duration):
    318     return datetime_obj.as_timedelta()

File /usr/local/lib/python3.10/site-packages/pendulum/parser.py:30, in parse(text, **options)
     26 def parse(text: str, **options: t.Any) -> Date | Time | DateTime | Duration:
     27     # Use the mock now value if it exists
     28     options["now"] = options.get("now")
---> 30     return _parse(text, **options)

File /usr/local/lib/python3.10/site-packages/pendulum/parser.py:43, in _parse(text, **options)
     40 if text == "now":
     41     return pendulum.now()
---> 43 parsed = base_parse(text, **options)
     45 if isinstance(parsed, datetime.datetime):
     46     return pendulum.datetime(
     47         parsed.year,
     48         parsed.month,
   (...)
     54         tz=parsed.tzinfo or options.get("tz", UTC),
     55     )

File /usr/local/lib/python3.10/site-packages/pendulum/parsing/__init__.py:78, in parse(text, **options)
     75 _options: dict[str, Any] = copy.copy(DEFAULT_OPTIONS)
     76 _options.update(options)
---> 78 return _normalize(_parse(text, **_options), **_options)

File /usr/local/lib/python3.10/site-packages/pendulum/parsing/__init__.py:116, in _parse(text, **options)
    113     return parse_iso8601(text)
    115 with contextlib.suppress(ValueError):
--> 116     return _parse_iso8601_interval(text)
    118 with contextlib.suppress(ParserError):
    119     return _parse_common(text, **options)

File /usr/local/lib/python3.10/site-packages/pendulum/parsing/__init__.py:217, in _parse_iso8601_interval(text)
    214 first, last = text.split("/")
    215 start = end = duration = None
--> 217 if first[0] == "P":
    218     # duration/end
    219     duration = parse_iso8601(first)
    220     end = parse_iso8601(last)

IndexError: string index out of range

The field that is being deserialised is 'WorkPhone' with a value of '/'. That is clearly not a date even though it is trying to decode it as a date.

Expected behavior

I expect to retrieve the decoded object

How to reproduce

from azure.identity import ClientSecretCredential
from msgraph import GraphServiceClient
import asyncio
from msgraph.generated.users.item.messages.messages_request_builder import  MessagesRequestBuilder

credentials = ClientSecretCredential(
            tenant_id=SHAREPOINT_TENANT_ID,
            client_id=SHAREPOINT_CLIENT_ID,
            client_secret=SHAREPOINT_SECRET_VALUE
        )
client = GraphServiceClient(credentials, scopes=["https://graph.microsoft.com/.default"])
loop = asyncio.get_event_loop()

site_id = <the_site_id>

query_params = MessagesRequestBuilder.MessagesRequestBuilderGetQueryParameters(expand=['fields'])
configuration = MessagesRequestBuilder.MessagesRequestBuilderGetRequestConfiguration(query_parameters=query_params)
fn = client.sites.by_site_id(site_id)lists.by_list_id("User Information List").items.get(request_configuration=configuration)
a = asyncio.run_coroutine_threadsafe(fn, loop)
a.result()

This obviously triggers the issue only when the data coming from the request contains the mentioned field with the specific value.

SDK Version

1.11.0

Latest version known to work for scenario above?

None

Known Workarounds

No response

Debug output

No response

Configuration

No response

Other information

No response

After several tests, the last configuration that works is the following:

msgraph-core==1.1.6
msgraph-sdk==1.5.3
microsoft-kiota-abstractions==1.5.0
microsoft-kiota-authentication-azure==1.5.0
microsoft-kiota-http==1.5.0
microsoft-kiota-serialization-form==1.5.0
microsoft-kiota-serialization-json==1.2.0
microsoft-kiota-serialization-multipart==1.5.0
microsoft-kiota-serialization-text==1.5.0

with microsoft-kiota-serialization-json==1.2.0 pinned to 1.2.0

Nicolas Lurkin (@nicolas-lurkin-b12)

Hello Nicolas Lurkin (@nicolas-lurkin-b12) thanks for using the SDK and for raising this.

This looks like an issue we have been experiencing with our JSON serialization:
If I get you right, you mean the latest versions do not work for your scenario and only works for 1.2.0?

Shem Ogumbe (@shemogumbe)

Hello Shem Ogumbe (@shemogumbe)
Thanks for your answer. That is correct. As far as I could tell, using microsoft-kiota-serialization-json above 1.2.0 leads to the issue.

Nicolas Lurkin (@nicolas-lurkin-b12)

I encountered possibly the same issue here.

Main Logic

async def list_outlook_events(self, start_date: datetime, end_date: datetime):
    """
    Lists events created or participated in by the user between start_date and end_date.
    Ref: https://learn.microsoft.com/en-us/graph/api/event-get?view=graph-rest-1.0
    """
    events = []
    request_configuration = RequestConfiguration()
    request_configuration.headers.add("Prefer", 'outlook.timezone="Asia/Taipei"')

    # Instead of RequestConfiguration, manually set headers in the query
    result = await self.graph_client.me.events.get(
        request_configuration=request_configuration
    )
    old_event_count = 0
    stop_processing = False

    while True:
        for event in result.value:
            try:
                # transformation logic
            except Exception as e:
                print(f"Error processing event: {e}")
                traceback.print_exc()
                pyperclip.copy(str(event))
                sys.exit(0)

        if stop_processing:
            break

        # Handle pagination if there are more events to retrieve
        if result.odata_next_link:
            result = await self.graph_client.teams.with_url(result.odata_next_link).get()

    .
    .
    .
    return events

I have 2 MacBook Pros, that said, MacBook 1 and MacBook 2. The Python script on MacBook 1 works pretty much well and MacBook 2 the same UNTIL TODAY, i.e., MacBook 1 good job and MacBook 2 wanted to play a game with me, a.k.a, failed.

Error Message from Traceback

Traceback (most recent call last):
  File "/Users/shu-jenghsieh/Desktop/Automation Center/journal/jira/metrics/collect_stats.py", line 506, in <module>
    asyncio.run(collect_calendar_events(input_year, all=True))
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.13/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ~~~~~~~~~~^^^^^^
  File "/opt/homebrew/Cellar/python@3.13/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/opt/homebrew/Cellar/python@3.13/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 721, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/Users/shu-jenghsieh/Desktop/Automation Center/journal/jira/metrics/collect_stats.py", line 477, in collect_calendar_events
    events = await lister.list_outlook_events(start_date_taipei, end_date_taipei)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shu-jenghsieh/Desktop/Automation Center/journal/jira/metrics/teams/list_outlook_events.py", line 546, in list_outlook_events
    result = await self.graph_client.teams.with_url(result.odata_next_link).get()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.13/site-packages/msgraph/generated/teams/teams_request_builder.py", line 69, in get
    return await self.request_adapter.send_async(request_info, TeamCollectionResponse, error_mapping)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.13/site-packages/kiota_http/httpx_request_adapter.py", line 196, in send_async
    value = root_node.get_object_value(parsable_factory)
  File "/opt/homebrew/lib/python3.13/site-packages/kiota_serialization_json/json_parse_node.py", line 222, in get_object_value
    self._assign_field_values(result)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/opt/homebrew/lib/python3.13/site-packages/kiota_serialization_json/json_parse_node.py", line 291, in _assign_field_values
    field_deserializer(JsonParseNode(field_value))
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.13/site-packages/msgraph/generated/models/team_collection_response.py", line 40, in <lambda>
    "value": lambda n : setattr(self, 'value', n.get_collection_of_object_values(Team)),
                                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/opt/homebrew/lib/python3.13/site-packages/kiota_serialization_json/json_parse_node.py", line 170, in get_collection_of_object_values
    return list(
        map(
    ...<2 lines>...
        )
    )
  File "/opt/homebrew/lib/python3.13/site-packages/kiota_serialization_json/json_parse_node.py", line 172, in <lambda>
    lambda x: self._create_new_node(x).get_object_value(factory),  # type: ignore
              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/homebrew/lib/python3.13/site-packages/kiota_serialization_json/json_parse_node.py", line 222, in get_object_value
    self._assign_field_values(result)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/opt/homebrew/lib/python3.13/site-packages/kiota_serialization_json/json_parse_node.py", line 293, in _assign_field_values
    item_additional_data[field_name] = self.try_get_anything(field_value)
                                       ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.13/site-packages/kiota_serialization_json/json_parse_node.py", line 316, in try_get_anything
    datetime_obj = pendulum.parse(value)
  File "/opt/homebrew/lib/python3.13/site-packages/pendulum/parser.py", line 30, in parse
    return _parse(text, **options)
  File "/opt/homebrew/lib/python3.13/site-packages/pendulum/parser.py", line 43, in _parse
    parsed = base_parse(text, **options)
  File "/opt/homebrew/lib/python3.13/site-packages/pendulum/parsing/__init__.py", line 78, in parse
    return _normalize(_parse(text, **_options), **_options)
                      ~~~~~~^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.13/site-packages/pendulum/parsing/__init__.py", line 116, in _parse
    return _parse_iso8601_interval(text)
  File "/opt/homebrew/lib/python3.13/site-packages/pendulum/parsing/__init__.py", line 221, in _parse_iso8601_interval
    elif last[0] == "P":
         ~~~~^^^
IndexError: string index out of range

In the beginning, I tweaked try_get_anything in the module of json_parse_node to see what field cannot be parsed and there are quite a lot and NOT every (Outlook) fetching from odata_next_link failed yet still quite some link couldn't return results correctly. By conjecturing some issue in JSON parsing and asking Teach Goo (Google), I saw this discussion. Thanks to Nicolas Lurkin (@nicolas-lurkin-b12) , my issue is resolved by adding microsoft-kiota-serialization-json==1.3.0 in requirements.txt in my project for automation.

I have tried the following versions for microsoft-kiota-serialization-json

microsoft-kiota-serialization-json==1.3.1
microsoft-kiota-serialization-json==1.3.2
microsoft-kiota-serialization-json==1.3.3
microsoft-kiota-serialization-json==1.5.0
microsoft-kiota-serialization-json==1.6.0

And all led me to the same error message.

Machine Info

ProductName:            macOS
ProductVersion:         14.6.1
BuildVersion:           23G93

Python Version

Python 3.13.0

It's installed by brew on the laptop where I encountered the issue.

Hopefully, my experience can help more developers.
cc Shem Ogumbe (@shemogumbe)

I'm experiencing the same problem with my Airflow DAG that connects to OneDrive file with Graph SDK.

What is confusing in my case is that I have multiple similar DAGs connecting to OneDrive, but only one of them has this error.
Also if I run the same pipeline manually on Windows, it works.

Our setup:
Ubuntu 22.04.5 LTS

microsoft-kiota-abstractions==1.6.0
microsoft-kiota-authentication-azure==1.0.0
microsoft-kiota-http==1.3.1
microsoft-kiota-serialization-form==1.6.0
microsoft-kiota-serialization-json==1.6.0
microsoft-kiota-serialization-multipart==1.6.0
microsoft-kiota-serialization-text==1.6.0

Python version:
Python 3.12.4

On Windows:
microsoft-kiota-serialization-json==1.3.0

After upgrading from microsoft_kiota_serialization_json-1.4.5 to microsoft_kiota_serialization_json-1.9.2, I could confirm that the issue is resolved.

This issue stems from pendulum library accessing out of range string indices. (Fixed in this MR: python-pendulum/pendulum#843 , yet to be released by upstream)

The library, pendulum, has been removed in this MR (microsoft/kiota-python@8a66dc4 , released).

Testing with SharePoint ListItem API, it was found that the test case of / value no longer produce error, and the issue should therefore be considered fixed.

lifegivesyoulemons

lifegivesyoulemons https://touching-notch53.github.io/blQllthpM4j3XJyTCieZgeqF6d44nyowGX96USMznqd_q7WwhRJJ3wfDqg/

Navigation Menu

Repository navigation

Exception: string index out of range when fetching site lists with expand:fields #940

Describe the bug

Expected behavior

How to reproduce

SDK Version

Latest version known to work for scenario above?

Known Workarounds

Debug output

Configuration

Other information

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Exception: string index out of range when fetching site lists with expand:fields #940

Description

Describe the bug

Expected behavior

How to reproduce

SDK Version

Latest version known to work for scenario above?

Known Workarounds

Debug output

Configuration

Other information

Activity

nicolas-lurkin-b12 commented on Oct 24, 2024

shemogumbe commented on Oct 26, 2024

nicolas-lurkin-b12 commented on Oct 28, 2024

HsiehShuJeng commented on Oct 31, 2024

lifegivesyoulemons commented on Nov 12, 2024

saengowp commented on Feb 24, 2025

Percydisruption61 commented on Oct 17, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions