Understand it better

[pavittarx]

Lesson: https://github.com/microsoft/ML-For-Beginners/tree/main/2-Regression/1-Tools

I have gone thorough the lesson in the last week a couple of times. Yet, I have some questions or doubts, that I may need some clarity upon.

    X = X[:, np.newaxis, 2]

How does this specific line work? I have gone through the some of the numpy library, tried searching on Github. But have not been able to find a satisfactory answers.

From what I got from Stack Overflow it tells that np.newaxis converts your array from n to n+1 th dimension. But, here it is being used to pluck a column from the dataset. How does this exactly work?

[abhi-bhatra]

Hi,
This line takes the second element from all the columns of array X and creates a new array for you with new dimensions.

1. When we initialize X, its shape was (442, 10)

2. But, when we use np.newaxis for array slicing, its shape will be (442, 1)

Let me explain you np.newaxis works with a simple example:

import numpy as np
arr = np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]])
print(arr)

Here the output will be:

[[ 1  2  3  4  5]
 [ 6  7  8  9 10]]

But, when you do np.newaxis slicing:

import numpy as np
arr = np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]])
print(arr[:, np.newaxis, 2])

Here, the output will be:

[[3]
 [8]]

I think the above shown example will help you better understand the use of np.newaxis in the following Model.

[pavittarx]

Thanks, for that explanation @abhi-bhatra

However, I wanted to know a bit more about this. Like say we can also slice values with X[:, 2:3] but why did we use np.newaxis.

Is is the same or are there any other advantages or something as such with this one?

Sorry, but I'm kinda new to Python. So, I am looking to have little deeper understanding.

[jlooper]

It's a valid question. It's borrowed from this example on Scikit-learn's docs: https://scikit-learn.org/stable/auto_examples/linear_model/plot_ols.html#sphx-glr-auto-examples-linear-model-plot-ols-py to choose a feature. The StackOverflow explanation below is pretty good: newaxis adds a dimension to your array, ensuring that it is 2D in shape as expected for the training procedure.

https://stackoverflow.com/questions/55500398/use-of-numpy-newaxis-in-machine-learning

[pavittarx]

I pondered over this line for a couple of days. Funny that, it was so simple of I just out so much for it.

[mubarak-mustopha]

I always get stuck whenever I'm trying do some data analysis before actually building a model.
Like I don't really know the right questions to ask based on the data I have.....can anyone make a recommendation or a resource that can help me solve this problem?

[mubarak-mustopha]

Hi @abhi-bhatra please check my GitHub repo out

[abhi-bhatra]

Hi @mubarak-mustopha Yes I am going to review it and will get back to you !!!

[mubarak-mustopha]

@abhi-bhatra thank you

[jlooper]

hi @mubarak-mustopha thanks for open sourcing these! I will take a look!

[mubarak-mustopha]

Hi @jlooper thank you.... I'll keep updating u guys once I add new commits

[mubarak-mustopha]

@jlooper @abhi-bhatra
I've added more commits to my repo,you guys should please check them out and let me know where I can improve https://github.com/mubarak-mustopha/microsoft_mlcourse

[abhi-bhatra]

@mubarak-mustopha It's great to see your enthusiastic spirit of learning ML. I will definitely look into it, and get back to you 👍

[mubarak-mustopha]

@abhi-bhatra thanks so much.

[mubarak-mustopha]

@jlooper @abhi-bhatra
In the time-series tutorial I'm getting an error 'No module named 'common' whenever i try importing common.utils, is there any set up I'm supposed to do before the import statement.