# What’s New in PyQ 4.0¶

Release: | 4.0.2 |
---|---|

Date: | May 12, 2017 |

## Summary – Release highlights¶

- Enhanced q) prompt with syntax highlighting.
- New operators:
`<<`

,`>>`

and`@`

.- Improved means for constructing
`K`

objects of arbitrary types.- Type casts using attribute syntax.
- Improved numpy interoperability.
- Restored support for KDB+ 2.x.
- Better documentation.
- More
`k.h`

functions are exposed to Python internally.- Added convenience scripts for starting different interactive sessions.
- Additional conversions between
`K`

and native Python objects.- Redesigned adverbs

## Enhanced `q)`

prompt¶

The `q)`

prompt will now use the prompt toolkit when available to provide
a separate command history, q syntax highlighting and a status bar displaying
system information.

## New operators¶

Three new operators are defined for `K`

objects: `<<`

, `>>`

and `@`

.

### Shift operators¶

Shift operators `<<`

and `>>`

can now be used to shift elements in
`K`

lists:

```
>>> q.til(10) << 3
k('3 4 5 6 7 8 9 0N 0N 0N')
>>> q.til(10) >> 3
k('0N 0N 0N 0 1 2 3 4 5 6')
```

### The `@`

operator¶

Users of Python 3.5 or later can now use the new binary operator `@`

to
call q functions without using parentheses:

```
>>> q.til @ 5
k('0 1 2 3 4')
```

The same operator between two functions creates a function composition. For example, the dot product can be defined succinctly as

```
>>> dot = q.sum @ q('*')
>>> dot([1, 2, 3], [3, 2, 1])
k('10')
```

## Typed constructors and casts¶

Atoms and lists of like atoms can now be constructed from Python objects
using typed constructors. For example, by default, a list of strings passed
to the default `K`

constructor becomes a symbol list:

```
>>> colors = K(['white', 'blue', 'red'])
>>> colors
k('`white`blue`red')
```

If you want to create a list of strings, you can use a typed constructor:

```
>>> K.string(["Donald E. Knuth", "Edsger W. Dijkstra"])
k('("Donald E. Knuth";"Edsger W. Dijkstra")')
```

If you already have a symbol list and want to convert it to strings, you can use the attribute access notation to perform the cast:

```
>>> colors.string
k('("white";"blue";"red")')
```

Similar operations can be performed with numeric data. For example, to create a matrix of single-precision floats (real), call

```
>>> m = K.real([[1, 0, 0],
... [0, 1, 0],
... [0, 0, 1]])
>>> m
k('(1 0 0e;0 1 0e;0 0 1e)')
```

To cast the result to booleans — access the `boolean`

attribute:

```
>>> m.boolean.show()
100b
010b
001b
```

Unlike q, Python does not have special syntax for missing values and infinities.
Those values can now be created in PyQ by accessing `na`

and `inf`

attributes
on the typed constructors:

```
>>> for x in [K.int, K.float, K.date, K.timespan]:
... print(x.na, x.inf)
0Ni 0Wi
0n 0w
0Nd 0Wd
0Nn 0Wn
```

## Interoperability with NumPy¶

### Matrices and arrays of higher dimensions¶

Arrays with `ndim > 1`

can now be passed to `q`

and they become nested
lists. For example:

```
>>> q.x = numpy.arange(12, dtype=float).reshape((2, 3, 2))
>>> q.x
k('((0 1f;2 3f;4 5f);(6 7f;8 9f;10 11f))')
```

Similarly, `ndim > 1`

arrays can be constructed from lists of regular shape:

```
>>> numpy.array(q.x)
array([[[ 0., 1.],
[ 2., 3.],
[ 4., 5.]],
[[ 6., 7.],
[ 8., 9.],
[ 10., 11.]]])
```

### Times, dates and timedeltas¶

Prior to 4.0, conversion of temporal data to NumPy arrays would expose internal integer values. For example, a list of months

```
>>> months = q('2001.01m + til 3')
```

would become an integer array when converted to NumPy:

```
>>> numpy.array(months).tolist()
[12, 13, 14]
```

Now, an array of type datetime64 is returned:

```
>>> numpy.array(months)
array(['2001-01', '2001-02', '2001-03'], dtype='datetime64[M]')
```

Note that the resulting array has different numeric values and cannot share the
data with the `K`

object. To share the data and/or to get an array
as in older versions, one should use the new `data`

attribute:

```
>>> a = numpy.asarray(months.data)
>>> a.tolist()
[12, 13, 14]
```

An array constructed from the `data`

attribute will use the same
underlying storage. This means that changing the array will change the
`K`

object.

```
>>> a[:] += 998*12
>>> months
k('2999.01 2999.02 2999.03m')
```

## Additional conversions¶

### Complex numbers¶

Complex numbers can now be passed to and obtained from kdb+. When passed to kdb+, complex numbers are automatically converted to dictionaries with keys “re” and “im” and lists of complex numbers are converted to tables with columns “re” and “im”.

```
>>> q.z = [1 + 2j, 3 + 4j, 5 + 6j]
>>> q.z.show()
re im
-----
1 2
3 4
5 6
>>> [complex(x) for x in q.z]
[(1+2j), (3+4j), (5+6j)]
```

### Path objects¶

`Path`

objects can now be used where q path handle symbols
are expected

```
>>> import pathlib
>>> path = pathlib.Path('xyz')
>>> q.set(path, 42)
k('`:xyz')
>>> q.get(path)
k('42')
```

### Named tuples¶

Named tuples are now converted to dictionaries:

```
>>> from collections import namedtuple
>>> Point = namedtuple('Point', 'x,y')
>>> q.point = Point(1, 2)
>>> q.point
k('`x`y!1 2')
```

As a consequence, a uniform list of named tuples is converted to a table:

```
>>> q.points = [Point(1, 2), Point(3, 4), Point(5, 6)]
>>> q.points.show()
x y
---
1 2
3 4
5 6
```

## Redesigned adverbs¶

Adverbs can now be used on functions with different ranks. For example,
`scan`

and `over`

can be used with monadic functions. To illustrate,
the following code generates a Pascal triangle:

```
>>> f = q('{(0,x)+x,0}')
>>> f.scan(6, 1).show()
1
1 1
1 2 1
1 3 3 1
1 4 6 4 1
1 5 10 10 5 1
1 6 15 20 15 6 1
```

If only the last row is of interest – use `over`

:

```
>>> f.over(6, 1)
k('1 6 15 20 15 6 1')
```