Make str and bytes error messages on concatenation conform with other sequences

@rhettinger

BPO	29116
Nosy	@rhettinger, @terryjreedy, @ned-deily, @serhiy-storchaka, @ilevkivskyi, @zhangyangyu, @DimitrisJim
PRs	bpo-29116: Fix error messages for concatenating bytes and bytearray w… #709 bpo-29116: Improve error message for concatenating str with non-str. #710 bpo-29116: Fix error messages for concatenating bytes and bytearray w… #723 bpo-29116: Fix error messages for concatenating bytes and bytearray w… #724

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-02-19.12:11:02.111>
created_at = <Date 2016-12-30.18:56:51.731>
labels = ['interpreter-core', 'type-feature', '3.7']
title = 'Make str and bytes error messages on concatenation conform with other sequences'
updated_at = <Date 2018-02-19.12:11:02.110>
user = 'https://github.com/DimitrisJim'

bugs.python.org fields:

activity = <Date 2018-02-19.12:11:02.110>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2018-02-19.12:11:02.111>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2016-12-30.18:56:51.731>
creator = 'Jim Fasarakis-Hilliard'
dependencies = []
files = []
hgrepos = []
issue_num = 29116
keywords = ['patch']
message_count = 18.0
messages = ['284340', '284341', '284348', '284370', '284372', '284545', '284855', '284861', '285213', '288922', '289803', '289866', '289877', '290151', '290152', '290158', '290159', '304958']
nosy_count = 7.0
nosy_names = ['rhettinger', 'terry.reedy', 'ned.deily', 'serhiy.storchaka', 'levkivskyi', 'xiang.zhang', 'Jim Fasarakis-Hilliard']
pr_nums = ['709', '710', '723', '724']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue29116'
versions = ['Python 3.7']

Specifically, bytes (always, from what I could find) had this error message:

    >>> b'' + ''
    TypeError: can't concat bytes to str

while str, after a change in issue26057, was made to:

    >>> '' + b''
    TypeError: must be str, not bytes

from the previous form of "TypeError: Can't convert 'bytes' object to str implicitly".

I think these could be changed to conform with what the other sequences generate and fall in line with the error messages produced for other operations.

Specifically changing them to:

    >>> b'' + ''
    TypeError: can only concatenate bytes (not 'str') to bytes

and similarly for str.

If this idea makes sense, I can attach a patch that addresses it.

Old error message is misleading. It implied that there are objects that can be converted to str implicitly. This was true in Python 2, but is false in Python 3.

New error message conforms with TypeError messages produced by PyArg_Parse*(). It is the same in a number of str methods. It doesn't conform with the error messages for other produced by concatenating other types. But different messages are generated for different types:

>>> b'' + 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't concat bytes to int
>>> a = bytearray(); a += 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't concat int to bytearray
>>> [] + 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can only concatenate list (not "int") to list
>>> a = []; a += 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not iterable
>>> import array; array.array('b') + []
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can only append array (not "list") to array
>>> import array; a = array.array('b'); a += []
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can only extend array with array (not "list")
>>> import operator; operator.concat(1, '')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object can't be concatenated

I think it would be better to unify them all.

I'm not sure that this change can be considered as a bug fix rather than an enhancement. Leave this on 3.6 release manager.

Should that message be the one predominantly used for sequences, i.e:

TypeError: can only concatenate class1 (not "class2") to class1

or should another one be used like "Unsupported operand type(s) for op: 'class1' and 'class2'?

The first is problematic with cases like += where an iterable is accepted, the second seems better to me at least.

As for operator.concat, any reason why the check is made beforehand to see if the first argument has a __getitem__ method? Couldn't that just be removed allowing the exception from concat(1, '') to just propagate to the caller?

Different cases accept different types. Not always we can say that only one specific type is accepted.

"list +" accepts only lists, "list +=" is a syntax sugar for list.extend and accepts any iterables. "bytes +" and "bytearray +=" accept any objects that support the buffer protocol. "array +" and "array +=" accept only arrays.

Changing semantic is out of the scope of this issue. I think there are reasons for current behavior.

Originally this issue was raised on StackOverflow:

http://stackoverflow.com/questions/41388606/python-3-6-vs-3-5-typeerror-message-on-string-concatenation

As for operator.concat, any reason why the check is made beforehand to see if the first argument has a __getitem__ method?

Concatenation is an operation of sequence (+ could be used both for concatenation and addition), so you have to check __getitem__ to make sure it's a sequence.

Couldn't that just be removed allowing the exception from concat(1, '') to just propagate to the caller?

If removed, concat(1, '') could propagate exceptions but not concat(1, 1).

By default, error message wording changes are 'enhancements' that wait for the next x.y.0 release unless the current wording is positively wrong'. This is different from doc changes because there are tests depending on error messages. 'Inconsistent' or 'awkward' is not the same as 'wrong'.

I do agree that the clipped "must be str, not bytes' is awkward. What (which) is it that must be str? A fleshed out and positive "can only concatenate str (not bytes) to str." would be clearer and educate users better.

As Serhiy said, the exact equivalent cannot be said for bytes.

>>> b'a' + bytearray(b'b')
b'ab'
>>> b'a' + memoryview(b'b')
b'ab'

However "Can only concatenate bytes, bytearray, or memoryview (not x) to bytes." would be good if this is in fact complete. I need to be educated on this ;-) I was not sure about memoryview until I tried it.

You can concatenate any object supporting the buffer protocol to bytes and bytearray.

Current error message for bytes looks awkward too, because it says "bytes to othertype" instead of "othertype to bytes".

As I currently see this:

The error message for str can be changed to the one used for other sequences 'can only concatenate str (not "type") to str'
The error message for arrays can be changed to use concatenate instead of append, too.

For bytes I see a conundrum, on one hand it can be changed to mention the buffer protocol which might confuse new users more than it helps them or, it can try and mention the objects that currently conform to it which might be exhaustive and long.

I'm removing myself as assignee as this doesn't seem to need a 3.6 RM decision at this point. If necessary, we can discuss a 3.6 backport after the issue has been resolved for 3.7.

PR 709 fixes the order of types for concatenating bytes and and bytearray. b''+0 will raise "can't concat str to bytes" rather than "can't concat bytes to str".

PR 710 makes error message for str concatenation more informative and similar to error messages for list, tuple, deque, array. '' + b'' will raise "can only concatenate str (not "bytes") to str" rather than "must be str, not bytes".

Something is strange: PRs 709, 723, 724 are shown as open in the "Pull Requests" section on this page. However, all four PRs are already merged.

Are other see the same? Shouldn't status be automatically updated?

I don't know why statuses was not updated automatically. Updated them manually.

Ivan in the comment on GitHub suggested to use "concatenate" instead of "concat" in "can't concat <type> to bytes". Maybe make it more similar to messages for list, tuple, deque, array? "can only concatenate objects supporting the buffer protocol (not "<type>") to bytes"?

New changeset 2c5b2c3 by Serhiy Storchaka in branch '3.5':
bpo-29116: Fix error messages for concatenating bytes and bytearray with unsupported type. (#709) (#724)
2c5b2c3

New changeset 3d258b1 by Serhiy Storchaka in branch '3.6':
bpo-29116: Fix error messages for concatenating bytes and bytearray with unsupported type. (#709) (#723)
3d258b1

New changeset 6b5a9ec by Serhiy Storchaka in branch 'master':
bpo-29116: Fix error messages for concatenating bytes and bytearray with unsupported type. (#709)
6b5a9ec

New changeset 004e03f by Serhiy Storchaka in branch 'master':
bpo-29116: Improve error message for concatenating str with non-str. (#710)
004e03f

In the future, I don't think this sort of thing should be backported. It isn't a bug, rather, it is a "thing x doesn't exactly look like thing y".

Navigation Menu

Repository navigation

Make str and bytes error messages on concatenation conform with other sequences #73302

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Make str and bytes error messages on concatenation conform with other sequences #73302

Description

Activity

DimitrisJim commented on Dec 30, 2016

serhiy-storchaka commented on Dec 30, 2016

DimitrisJim commented on Dec 30, 2016

serhiy-storchaka commented on Dec 31, 2016

serhiy-storchaka commented on Dec 31, 2016

zhangyangyu commented on Jan 3, 2017

terryjreedy commented on Jan 6, 2017

serhiy-storchaka commented on Jan 6, 2017

DimitrisJim commented on Jan 11, 2017

ned-deily commented on Mar 3, 2017

serhiy-storchaka commented on Mar 18, 2017

ilevkivskyi commented on Mar 19, 2017

serhiy-storchaka commented on Mar 20, 2017

serhiy-storchaka commented on Mar 24, 2017

serhiy-storchaka commented on Mar 24, 2017

serhiy-storchaka commented on Mar 24, 2017

serhiy-storchaka commented on Mar 24, 2017

rhettinger commented on Oct 25, 2017

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions