In this article, I am going to explain how the “reshape” function works. First, I will start with an explanation of the array shapes. Later, I will show you how to use the reshape function properly. In the end, I will demonstrate the infamous -1 shape, an array which contains many observations of only one feature, and an array with one observation of multiple features).

Shape

One dimension

First of all, we must know that the shape of an array is always a tuple. That is why sometimes we see a weird shape like (10,) which mean that we have a one-dimensional array of 10 elements. This strange notation is simply the Python way of writing a tuple containing only one element.

Let’s look at an example. I created a one-dimensional array of 10 elements, and as I wrote in the previous paragraph, the shape of such array is (10,)

arr = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
arr.shape
# (10,)

Many dimensions

When I create a multi-dimensional array, the first number in the tuple denotes the length of the outermost array. The following numbers are lengths of the subsequent nested arrays. The number of values in the tuple denotes the number of dimensions (nesting levels) of the arrays.

In the case of the two-dimensional array, I have the following values:

arr = np.array([[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]])
arr.shape
# (1, 10)

(1,10) means that the outer array consists of one nested array, and the inner array has 10 elements.

In the next example, I create a three-dimensional array in which the outer array has three nested arrays, the middle array has five values, and every one of those values is another array consisting of two elements.

arr = np.array([
    [[1, 2], [1, 2], [1, 2], [1, 2], [1, 2]],
    [[1, 2], [1, 2], [1, 2], [1, 2], [1, 2]],
    [[1, 2], [1, 2], [1, 2], [1, 2], [1, 2]],
])
arr.shape
# (3, 5, 2)

As expected, the shape of such an array is (3, 5, 2) because the order of values in the tuple is the same as the nesting level of arrays.

Flattening arrays

Before we start changing the shape of arrays, we must get familiar with the “ravel” function which returns a “flattened” representation of an array. It is essential because when we reshape an array, the reshape function is first going to flatten the input, and then split it into new arrays.

Flattened means that we get rid of all squared brackets and return the array elements one by one enclosed in a single array.

If I flatten this array: [[1, 2, 3], [4, 5, 6]] I get an array of ordered numbers between 1 and 6.

arr = np.array([[1, 2, 3], [4, 5, 6]])
arr.ravel()
# array([1, 2, 3, 4, 5, 6])

Obviously, flattening a single-dimensional array does not do anything.

arr = np.array([1, 2, 4, 8, 16, 32])
arr.ravel()
# array([ 1,  2,  4,  8, 16, 32])

Reshape

Now, we know everything we need to use the reshape function correctly. First, I am going to define an array which we will use for testing. It is a three-dimensional array whose current shape is (4, 5, 3).

example = np.array([
    [[1, 2, 3], [0.5, 4, 6], [0.25, 8, 9], [0.125, 16, 12], [0.625, 32, 15]],
    [[3, 6, 9], [1.5, 12, 18], [0.75, 24, 27], [0.375, 48, 36], [1.875, 96, 45]],
    [[5, 10, 15], [2.5, 20, 30], [1.25, 40, 45], [0.625, 80, 60], [3.125, 160, 75]],
    [[0.25, 0.5, 0.75], [0.125, 1, 1.5], [0.0625, 2, 2.25], [0.03125, 4, 3], [0.15625, 8, 3.7]]
])
example.shape
# (4, 5, 3)

We can encounter such array while building a machine learning model for time series forecasting. In such a case, the outer array contains observations (examples), the middle array is the time-series (5 data points recorded in a different time but all belong to the same observations), the innermost array contains the features.

If I want to use such an array to train a machine learning model that works only with two-dimensional training data, I need to convert it into an array of four observations and 15 features (in this case we call it “lag features” because my array contains many measurements of the same features at a different time).

I can do it by calling the reshape function and giving it the target shape (4, 15)

example.reshape((4, 15))
# array([[  1.     ,   2.     ,   3.     ,   0.5    ,   4.     ,   6.     ,
#          0.25   ,   8.     ,   9.     ,   0.125  ,  16.     ,  12.     ,
#          0.625  ,  32.     ,  15.     ],
#       [  3.     ,   6.     ,   9.     ,   1.5    ,  12.     ,  18.     ,
#          0.75   ,  24.     ,  27.     ,   0.375  ,  48.     ,  36.     ,
#          1.875  ,  96.     ,  45.     ],
#       [  5.     ,  10.     ,  15.     ,   2.5    ,  20.     ,  30.     ,
#          1.25   ,  40.     ,  45.     ,   0.625  ,  80.     ,  60.     ,
#          3.125  , 160.     ,  75.     ],
#       [  0.25   ,   0.5    ,   0.75   ,   0.125  ,   1.     ,   1.5    ,
#          0.0625 ,   2.     ,   2.25   ,   0.03125,   4.     ,   3.     ,
#          0.15625,   8.     ,   3.7    ]])

We see that reshape with the shape parameter (4, 15) has first flattened the input array. Then it split the flattened array into four arrays of 15 elements and returned those arrays as a nested two-dimensional structure.

Shape -1

In the previous example, I didn’t need to set the length of the inner array. If I use the -1 value as one of the shapes, numpy will calculate the correct number for me automatically.

example.reshape((4, -1)).shape
# (4, 15)

Note that only one of the shapes can be automatically inferred because in numpy the inferred shape = the total length of the flattened array / the sum of given shapes.

Arrays of observations and their features

When we train a machine learning model, the nesting levels of arrays have precisely defined meaning. The external array contains observations/rows. The inner array contains columns/features.

This causes two special cases when we have either an array of multiple observations of only one feature or a single observation of multiple features.

Single observation arrays (1, -1)

Let’s start with a one-dimensional array. We want to use such an array to train a machine learning model, but for training, we need two-dimensional array even if one of those dimensions has length equal 1.

example = np.array([1, 2, 3, 4, 5])
example.shape
# (5,)

If we know that the one-dimensional array contains the values of multiple features, but only one observation, we can transform it into the proper shape by calling the reshape function with parameter (1, -1).

As always, the first element of the tuple is the length of the external array, which contains observations. I have one observation, so I wrote 1. The length of the inner array (the features) can be inferred automatically. Hence -1.

example.reshape((1, -1))
#array([[1, 2, 3, 4, 5]])
#shape: (1, 5)

Single feature arrays (-1, 1)

If the array used in the previous example contains many observations of one feature, I can transform it into a two-dimensional array by using (-1, 1) as the argument of the reshape function.

example.reshape((-1, 1))
#array([[1],
#       [2],
#       [3],
#       [4],
#       [5]])
#shape: (5, 1)

In this case, I wanted to infer the number of observations, but I know that every one of them has only one feature, so the inner array has size 1.

Numpy reshape explained

Shape

One dimension

Many dimensions

Flattening arrays

Reshape

Shape -1

Arrays of observations and their features

Single observation arrays (1, -1)

Single feature arrays (-1, 1)

Human bias in A/B testing

How to write to a Parquet file in Python

Numpy reshape explained

Shape

One dimension

Many dimensions

Flattening arrays

Reshape

Shape -1

Arrays of observations and their features

Single observation arrays (1, -1)

Single feature arrays (-1, 1)

Human bias in A/B testing

How to write to a Parquet file in Python

Related Posts

I worked as a data scientist and that was the worst job I have ever had.

How to remove outliers from Seaborn boxplot charts

Pandas stack and unstack explained