python function signatures are flexible, complex beasts, allowing for
positional, keyword, variable, and variable keyword arguments (and
parameters). This can be extremely useful, but sometimes the
intersection between these features can be confusing or even
surprising, especially on Python 2. What do you expect this to return?
>>> def test(arg1, **kwargs):
... return arg1
>>> test(**{'arg1': 42})
...
Contents
Terminology
Surprises
Non-Default Parameters Accept Keyword Arguments
Corollary: Variable Keyword Arguments Can Bind Non-Default Parameters
Corollary: Positional Parameters Consume Keyword Arguments
Default Parameters Accept Positional Arguments
Corollary: Variable Positional Arguments Can Bind Default Parameters
Mixing Variable Parameters and Keyword Arguments Will Break
Functions Implemented In C Can Break The Rules
Python 3 Improvements
Terminology
Before we move on, let's take a minute to define some terms.
parameter
The name of a variable listed by a function definition. These are
sometimes called "formal parameters" or "formal arguments." In def foo(a, b)
, a
and b
are parameters.
argument
The expression given to a function application (function
call). In foo(1, "str")
, 1
and "str"
are arguments.
function signature
The set of parameters in a function definition. This is also known
as a "parameter list."
binding
The process of associating function call arguments
to the parameter
names given in the function's signature. In foo(1, "str")
,
the parameter a
will be assigned the value 1
, and the
parameter b
will be assigned the value "str"
. This is also
called "argument filling."
default parameter
In a function signature, a parameter that is assigned a value.
An argument for this parameter does not have to be given in a function
application; when it is not, the default value is bound to the
parameter. In def foo(a, b=42)
, b=42
creates a default
parameter. It can also be said that b
has a default parameter
value. The function can be called as foo(1).
positional argument
An argument in a function call that's given in order of the
parameters in the function signature, from left to right. In foo(1, 2)
, 1
and 2
are positional arguments that will
be bound to the parameters a
and b
.
keyword argument
An argument in a function call that's given by name, matching the
name of a parameter. In foo(a=1)
, a=1
is a keyword
argument, and the parameter a
will have the value 1
.
variable parameters
A function signature that contains *args
(where args
is an
arbitrary identifier) accepts an arbitrary number of unnamed
arguments in addition to any explicit parameters. Extra
parameters are bound to args
as a list. def
foo(a, b, *args)
creates a function that has variable
parameters, and foo(1, 2)
, foo(1, 2, 3)
, foo(1, 2, 3,
4)
are all valid ways to call it. This is commonly called
"varargs," for "variable arguments" (even though it is a parameter
definition).
variable positional arguments
Passing an arbitrary (usually unknown from the function call itself)
number of arguments to a function by unpacking a sequence.
Variable arguments can be given to a function whether or not it
accepts variable parameters (if it doesn't, the number of variable
arguments must match the number of parameters). This is done using
the *
syntax:
foo(*(1,
2))
is the same as writing foo(1,
2)
, but more often the arguments are created dynamically. For
example,
args = (1, 2) if full_moon else (3, 4); foo(*args)
.
variable keyword parameters
A function signature that contains **kwargs
(where kwargs
is an arbitrary identifier) accepts an arbitrary number of keyword
arguments in addition to any explicit parameters (with or without
default values). The definition def foo(a, b, **kwargs)
creates a function with a variable keyword parameter. It can be
called like foo(1, 2)
or foo(1, 2, c=42, d=420)
.
variable positional arguments
Similar to variable positional arguments
, but using keyword
arguments. The syntax is **
, and the object to be unpacked
must be a mapping; extra arguments are placed in a mapping bound
to the parameter identifier. A simple example is
foo(**{'b':"str",
'a':1})
.
Some language communities are fairly strict about the usage of these
terms, but the Python community is often fairly informal. This is
especially true when it comes to the distinction between parameters
and arguments
(despite it being a FAQ
)
which helps lead to some of the surprises we discuss below.
Surprises
On to the surprises. These will all come from the intersection of the
various terms defined above. Not all of these will surprise everyone,
but I
would be surprised if most people didn't discover at least one
mildly surprising thing.
Non-Default Parameters Accept Keyword Arguments
Any parameter can be called using a keyword argument, whether or not
it has a default parameter value:
>>> def test(a, b, c=42):
... return (a, b, c)
>>> test(1, 2)
(1, 2, 42)
>>> test(1, b='b')
(1, 'b', 42)
>>> test(c=1, b=2, a=3)
(3, 2, 1)
This is surprising because sometimes parameters with a default value
are referred to as "keyword parameters" or "keyword arguments,"
suggesting that only they can be called using a keyword argument. In
reality, the parameter just has a default value. It's the function
call site that determines whether to use a keyword argument or not.
One consequence of this: the parameter names of public functions, even
if they don't have a default, are part of the public signature of the
function. If you distribute a library, people can and will call
functions using keyword arguments for parameters you didn't expect
them to. Changing parameter names can thus break backwards
compatibility. (Below we'll see how Python 3 can help with this.)
Corollary: Variable Keyword Arguments Can Bind Non-Default Parameters
If we introduce variable keyword arguments, we see that this behaviour
is consistent:
>>> kwargs = {'a': 'akw', 'b': 'bkw'}
>>> test(**kwargs)
('akw', 'bkw', 42)
Corollary: Positional Parameters Consume Keyword Arguments
Knowing what we know now, we can answer the teaser from the beginning
of the article:
>>> def test(arg1, **kwargs):
... return arg1, kwargs
>>> test(**{'arg1': 42})
(42, {})
The named parameter arg1
, even when passed in a variable keyword
argument, is still bound by name. There are no extra arguments to
place in kwargs
.
Default Parameters Accept Positional Arguments
Any parameter can be called using a positional argument, whether or
not it has a default parameter value:
>>> def test(a=1, b=2, c=3):
... return (a, b, c)
>>> test('a', 'b', 'c')
('a', 'b', 'c')
This is the inverse of the previous surprise. It may be surprising for
the same reason, the conflation of keyword arguments and default
parameter values.
Of course, convention often dictates that default parameters are
passed using keyword arguments, but as you can see, that's not a
requirement of the language.
Corollary: Variable Positional Arguments Can Bind Default Parameters
Introducing variable positional arguments shows consistent behaviour:
>>> pos_args = ('apos', 'bpos')
>>> test(*pos_args)
('apos', 'bpos', 3)
Mixing Variable Parameters and Keyword Arguments Will Break
Suppose we'd like to define some parameters with default values
(expecting them to be passed as keyword arguments by convention), and
then also allow for some extra arguments to be passed:
>>> def test(a, b=1, *args):
... return (a, b, args)
The definition works. Now lets call it in some common patterns:
>>> test('a', 'b')
('a', 'b', ())
>>> test('a', 'b', 'c')
('a', 'b', ('c',))
>>> test('a', b=1)
('a', 1, ())
>>> test('a', b='b', *(1, 2))
Traceback (most recent call last):
...
TypeError: test() got multiple values for argument 'b'
As long as we don't mix keyword and variable (extra) arguments,
everything works out. But as soon as we mix the two, the variable
positional arguments are bound first, and then we have a duplicate
keyword argument left over for b
.
This is a common enough source of errors that, as we'll see below,
Python 3 added some extra help for it, and linters warn about it
.
Functions Implemented In C Can Break The Rules
We generally expect to be able to call functions with keyword
arguments, especially
when the corresponding parameters have default
values, and we expect that the order of keyword arguments doesn't
matter. But if the function is not implemented in Python, and instead
is a built-in function implemented in C, that may not be the case.
Let's look at the built-in function open
. On Python 3, if we ask
for the function signature, we get something like this:
>>> import math
>>> help(math.tan)
Help on built-in function tan in module math:
<BLANKLINE>
...
tan(x)
<BLANKLINE>
Return the tangent of x (measured in radians).
<BLANKLINE>
That sure looks like a parameter with a name, so we
expect to be able to call it with a keyword argument:
>>> math.tan(x=1)
Traceback (most recent call last):
...
TypeError: tan() takes no keyword arguments
This is due to how C functions bind Python arguments into C variables,
using functions like PyArg_ParseTuple
.
In newer versions of Python, this is indicated with a trailing /
in the function signature, showing that the preceding arguments
are positional only paramaters
.
(Note that Python has no syntax for this.)
>>> help(abs)
Help on built-in function abs in module builtins:
<BLANKLINE>
abs(x, /)
Return the absolute value of the argument.
Python 3 Improvements
Python 3 offers ways to reduce some of these surprising
characteristics. (For backwards compatibility, it doesn't actually
eliminate any of them.)
We've already seen that functions implemented in C can use new syntax
in their function signatures to signify positional-only arguments.
Plus, more C functions can accept keyword arguments for any arbitrary
parameter thanks to new functions and the use of tools like Argument
Clinic
.
The most important improvements, though, are available to Python
functions and are outlined in the confusingly named PEP 3102
:
Keyword-Only Arguments.
With this PEP, functions are allowed to define parameters that can
only be filled by keyword arguments. In addition, this allows
functions to accept both variable arguments and keyword arguments
without raising TypeError
.
This in done by simply moving the variable positional parameter before
any parameters that should only be allowed by keyword:
>>> def test(a, *args, b=42):
... return (a, b, args)
>>> test(1, 2, 3)
(1, 42, (2, 3))
>>> test(1, 2, 3, b='b')
(1, 'b', (2, 3))
>>> test(1, 2, 3, b='b', c='c')
Traceback (most recent call last):
...
TypeError: test() got an unexpected keyword argument 'c'
>>> test()
Traceback (most recent call last):
...
TypeError: test() missing 1 required positional argument: 'a'
What if you don't want to allow arbitrary unnamed arguments? In that
case, simply omit the variable argument parameter name:
>>> def test(a, *, b=42):
... return (a, b)
>>> test(1, b='b')
(1, 'b')
Trying to pass extra arguments will fail:
>>> test(1, 2, b='b')
Traceback (most recent call last):
...
TypeError: test() takes 1 positional argument but 2 positional arguments (and 1 keyword-only argument) were given
Finally, what if you want to require
certain parameters to be passed
by name? In that case, you can simply leave off the default value for
the keyword-only parameters:
>>> def test(a, *, b):
... return (a, b)
>>> test(1)
Traceback (most recent call last):
...
TypeError: test() missing 1 required keyword-only argument: 'b'
>>> test(1, b='b')
(1, 'b')
The above examples all produce SyntaxError
on Python 2. Much of
the functionality can be achieved on Python 2 using variable arguments
and variable keyword arguments and manual argument binding, but that's
slower and uglier than what's available on Python 3. Lets look at an
example of implementing the first function from this section in Python
2:
>>> def test(*args, **kwargs):
... "test(a, *args, b=42) -> tuple" # docstring signature for Sphinx
... # This raises an IndexError instead of a TypeError if 'a'
... # is missing; that's easy to fix, but it's a programmer error
... a = args[0]
... args = args[1:]
... b = kwargs.pop('b', 42)
... if kwargs:
... raise TypeError("Got extra keyword args %s" % (list(kwargs)))
... return (a, b, args)
>>> test(1, 2, 3)
(1, 42, (2, 3))
>>> test(1, 2, 3, b='b')
(1, 'b', (2, 3))
>>> test(1, 2, 3, b='b', c='c')
Traceback (most recent call last):
...
TypeError: Got extra keyword args ['c']
>>> test()
Traceback (most recent call last):
...
IndexError: tuple index out of range
![[LoliHouse] Princess-Session Orchestra - 15 [WebRip 1080p HEVC-10bit...](http://s2.loli.net/2025/04/09/QO618K72ytGZmDJ.webp)
