{"id":61,"date":"2022-05-19T18:45:08","date_gmt":"2022-05-19T22:45:08","guid":{"rendered":"https:\/\/pressbooks.bccampus.ca\/businessanalytics\/chapter\/using_sklearn\/"},"modified":"2023-01-05T13:40:21","modified_gmt":"2023-01-05T18:40:21","slug":"using_sklearn","status":"publish","type":"chapter","link":"https:\/\/pressbooks.bccampus.ca\/businessanalytics\/chapter\/using_sklearn\/","title":{"raw":"Using_Sklearn","rendered":"Using_Sklearn"},"content":{"raw":"
\r\n
\r\n
\r\n
This page comes from a Jupyter Notebook, located at https:\/\/github.com\/amygoldlist\/BusinessAnalytics\/tree\/main\/AI_management<\/a>\r\n<\/em><\/div>\r\n
\r\n
\r\n

Using Sklearn\u00b6<\/a><\/h2>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
\r\n
\r\n
In\u00a0[1]:<\/div>\r\n
\r\n
\r\n
\r\n
##We should start by loading any packages we will need, as well as any data.<\/span>\r\n\r\n## We can import a full package, but I'm just going to import some:<\/span>\r\n\r\n## sklearn things we need:<\/span>\r\n#from sklearn.neighbors import KNeighborsClassifier<\/span>\r\nfrom<\/span> sklearn<\/span> import<\/span> neighbors<\/span>\r\nfrom<\/span> sklearn.neighbors<\/span> import<\/span> *<\/span>\r\nfrom<\/span> sklearn.model_selection<\/span> import<\/span> train_test_split<\/span>\r\nfrom<\/span> sklearn<\/span> import<\/span> datasets<\/span>\r\n\r\n# numpy and friends<\/span>\r\nimport<\/span> numpy<\/span> as<\/span> np<\/span>\r\nimport<\/span> matplotlib.pyplot<\/span> as<\/span> plt<\/span>\r\nfrom<\/span> scipy<\/span> import<\/span> stats<\/span>\r\n%<\/span>matplotlib<\/span> inline\r\n\r\n## We don't need all of this, but it's good to get anything I need in the very first block.<\/span>\r\n<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
\r\n
\r\n
In\u00a0[2]:<\/div>\r\n
\r\n
\r\n
\r\n
## Let's start with the handwritten numbers that we've seen a lot of:<\/span>\r\n\r\ndigits<\/span> =<\/span> datasets<\/span>.<\/span>load_digits<\/span>()<\/span>\r\n\r\nX<\/span> =<\/span> digits<\/span>[<\/span>'data'<\/span>]<\/span>   # this is the data with each 8x8 image \"flattened\" into a length-64 vector.<\/span>\r\nY<\/span> =<\/span> digits<\/span>[<\/span>'target'<\/span>]<\/span> # these are the labels (0-9). <\/span>\r\n\r\n## Let's look at one number:<\/span>\r\n## I've chosen randomly, but set n to any number under 1000<\/span>\r\n\r\n\r\nn<\/span> =<\/span> 653<\/span>\r\n\r\nplt<\/span>.<\/span>imshow<\/span>(<\/span>digits<\/span>[<\/span>'images'<\/span>][<\/span>n<\/span>],<\/span> cmap<\/span>=<\/span>'Greys_r'<\/span>)<\/span>\r\nplt<\/span>.<\/span>title<\/span>(<\/span>'This is a <\/span>%d<\/span>'<\/span> %<\/span> digits<\/span>[<\/span>'target'<\/span>][<\/span>n<\/span>])<\/span>\r\n\r\nprint<\/span>(<\/span>\"The X values are a 64 digit string:\"<\/span>,<\/span> X<\/span>[<\/span>n<\/span>])<\/span>\r\nprint<\/span>(<\/span>\"and the y is the value of the number\"<\/span>,<\/span> Y<\/span>[<\/span>n<\/span>])<\/span>\r\n<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
\r\n
\r\n
\r\n
<\/div>\r\n
\r\n
The X values are a 64 digit string: [ 0.  0.  4. 10. 16. 16.  7.  0.  0.  3. 16. 13. 11. 16.  2.  0.  0.  1.\r\n  3.  0. 10.  9.  0.  0.  0.  0.  5.  8. 14. 15. 13.  0.  0.  0. 15. 16.\r\n 14. 12.  8.  0.  0.  0.  3. 12.  7.  0.  0.  0.  0.  0.  0. 15.  4.  0.\r\n  0.  0.  0.  0.  3. 14.  1.  0.  0.  0.]\r\nand the y is the value of the number 7\r\n<\/pre>\r\n<\/div>\r\n<\/div>\r\n
\r\n
<\/div>\r\n
\"image\"<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
\r\n
\r\n
In\u00a0[5]:<\/div>\r\n
\r\n
\r\n
\r\n
## The first step is to break the data into the training set, and the test set.  <\/span>\r\n## Luckily, sklearn has you covered:<\/span>\r\n\r\nX_train<\/span>,<\/span> X_test<\/span>,<\/span> Y_train<\/span>,<\/span> Y_test<\/span> =<\/span> train_test_split<\/span>(<\/span>X<\/span>,<\/span>Y<\/span>,<\/span> test_size<\/span>=<\/span>0.2<\/span>)<\/span>\r\n\r\n## test_size tells the algrithm that we want to reserve 20% for the test (or validation) set.  Try it!  <\/span>\r\n## Now redo it with a test set at 30%<\/span>\r\n<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
\r\n
<\/div>\r\n
\r\n
\r\n
Assignment Questions\u00b6<\/a><\/h2>\r\n
    \r\n \t
  1. How big is the full data set? How many numbers in training set? How many are held back in the test set? You should have 30% in your test set.<\/li>\r\n \t
  2. What is the L1 distance between X[15]<\/code> and X[25]<\/code>? (Manhattan)<\/li>\r\n \t
  3. What is the L2 distance between X[15]<\/code> and X[25]<\/code>? (Euclidean)<\/li>\r\n \t
  4. What is the L3 distance between X[15]<\/code> and X[25]<\/code>?<\/li>\r\n<\/ol>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
    \r\n
    \r\n
    In\u00a0[1]:<\/div>\r\n
    \r\n
    \r\n
    \r\n
    ## Solutions<\/span>\r\n<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
    \r\n
    \r\n
    In\u00a0[1]:<\/div>\r\n
    \r\n
    \r\n
    \r\n
    ## Lets do it!<\/span>\r\nfrom<\/span> sklearn.neighbors<\/span> import<\/span> *<\/span>\r\n\r\n## Run this to learn about the function:<\/span>\r\n?<\/span>KNeighborsClassifier\r\n\r\n## FYI: Minkowski distance means the Lp norm, so we can set p = 2 for Euclidean distance<\/span>\r\n<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
    \r\n
    \r\n
    In\u00a0[7]:<\/div>\r\n
    \r\n
    \r\n
    \r\n
    ## For now, I'm going to set K = 3, and use the Euclidean or L2 norm.  <\/span>\r\n\r\n#Pick a name for your model -> I'm going to use a random word: cupcake<\/span>\r\n\r\n#step 1: let python know what algorithm I'm using:<\/span>\r\ncupcake<\/span> =<\/span>neighbors<\/span>.<\/span>KNeighborsClassifier<\/span>(<\/span>n_neighbors<\/span>=<\/span>15<\/span>,<\/span> p<\/span> =<\/span>2<\/span> )<\/span>\r\n#step 2: Train the model on ONLY THE TRAINING DATA (fit!)<\/span>\r\ncupcake<\/span>.<\/span>fit<\/span>(<\/span>X_train<\/span>,<\/span>Y_train<\/span>)<\/span>\r\n\r\n\r\n##Now we have a well trained model!  But is it a good model?<\/span>\r\n## I'm going to look at the errors in BOTH my training and test sets.  <\/span>\r\n## We do this by predicting what Y will be for each set, than actually comparing<\/span>\r\n\r\nY_pred_train<\/span> =<\/span> cupcake<\/span>.<\/span>predict<\/span>(<\/span>X_train<\/span>)<\/span> \r\nY_pred<\/span> =<\/span> cupcake<\/span>.<\/span>predict<\/span>(<\/span>X_test<\/span>)<\/span>\r\n\r\n##record errors<\/span>\r\nerror_train<\/span>=<\/span>list<\/span>(<\/span>Y_train<\/span> ==<\/span> Y_pred_train<\/span>)<\/span>.<\/span>count<\/span>(<\/span>False<\/span>)<\/span>\/<\/span>len<\/span>(<\/span>Y_train<\/span>)<\/span>\r\nerror_test<\/span>=<\/span>list<\/span>(<\/span>Y_test<\/span> ==<\/span> Y_pred<\/span>)<\/span>.<\/span>count<\/span>(<\/span>False<\/span>)<\/span>\/<\/span>len<\/span>(<\/span>Y_test<\/span>)<\/span>\r\n\r\n##Look it over:<\/span>\r\nprint<\/span>(<\/span>\"The training error was: \"<\/span>,<\/span> error_train<\/span>)<\/span>\r\nprint<\/span>(<\/span>\"The testing error was: \"<\/span>,<\/span> error_test<\/span>)<\/span>\r\n<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
    \r\n
    \r\n
    \r\n
    <\/div>\r\n
    \r\n
    The training error was:  0.016701461377870562\r\nThe testing error was:  0.025\r\n<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
    \r\n
    <\/div>\r\n
    \r\n
    \r\n
    Your turn:\u00b6<\/a><\/h2>\r\nFor this week, we are going to turn a blind eye to the fact that we are experimenting on the test data. Try out different models! Look at different [latex]K[\/latex] values, and different [latex]p[\/latex] values to see if you can find something that works better than others. Can you automate this process using some loops or other tools?\r\n\r\nExplain your best model, and hand it in. Expect that random change may affect different models.\r\n\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
    \r\n
    \r\n
    In\u00a0[\u00a0]:<\/div>\r\n
    \r\n
    \r\n
    \r\n
     \r\n<\/pre>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>","rendered":"
    \n
    \n
    \n
    This page comes from a Jupyter Notebook, located at https:\/\/github.com\/amygoldlist\/BusinessAnalytics\/tree\/main\/AI_management<\/a>
    \n<\/em><\/div>\n
    \n
    \n
    Using Sklearn\u00b6<\/a><\/h2>\n<\/div>\n<\/div>\n<\/div>\n
    \n
    \n
    In\u00a0[1]:<\/div>\n
    \n
    \n
    \n
    ##We should start by loading any packages we will need, as well as any data.<\/span>\r\n\r\n## We can import a full package, but I'm just going to import some:<\/span>\r\n\r\n## sklearn things we need:<\/span>\r\n#from sklearn.neighbors import KNeighborsClassifier<\/span>\r\nfrom<\/span> sklearn<\/span> import<\/span> neighbors<\/span>\r\nfrom<\/span> sklearn.neighbors<\/span> import<\/span> *<\/span>\r\nfrom<\/span> sklearn.model_selection<\/span> import<\/span> train_test_split<\/span>\r\nfrom<\/span> sklearn<\/span> import<\/span> datasets<\/span>\r\n\r\n# numpy and friends<\/span>\r\nimport<\/span> numpy<\/span> as<\/span> np<\/span>\r\nimport<\/span> matplotlib.pyplot<\/span> as<\/span> plt<\/span>\r\nfrom<\/span> scipy<\/span> import<\/span> stats<\/span>\r\n%<\/span>matplotlib<\/span> inline\r\n\r\n## We don't need all of this, but it's good to get anything I need in the very first block.<\/span>\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
    \n
    \n
    In\u00a0[2]:<\/div>\n
    \n
    \n
    \n
    ## Let's start with the handwritten numbers that we've seen a lot of:<\/span>\r\n\r\ndigits<\/span> =<\/span> datasets<\/span>.<\/span>load_digits<\/span>()<\/span>\r\n\r\nX<\/span> =<\/span> digits<\/span>[<\/span>'data'<\/span>]<\/span>   # this is the data with each 8x8 image \"flattened\" into a length-64 vector.<\/span>\r\nY<\/span> =<\/span> digits<\/span>[<\/span>'target'<\/span>]<\/span> # these are the labels (0-9). <\/span>\r\n\r\n## Let's look at one number:<\/span>\r\n## I've chosen randomly, but set n to any number under 1000<\/span>\r\n\r\n\r\nn<\/span> =<\/span> 653<\/span>\r\n\r\nplt<\/span>.<\/span>imshow<\/span>(<\/span>digits<\/span>[<\/span>'images'<\/span>][<\/span>n<\/span>],<\/span> cmap<\/span>=<\/span>'Greys_r'<\/span>)<\/span>\r\nplt<\/span>.<\/span>title<\/span>(<\/span>'This is a <\/span>%d<\/span>'<\/span> %<\/span> digits<\/span>[<\/span>'target'<\/span>][<\/span>n<\/span>])<\/span>\r\n\r\nprint<\/span>(<\/span>\"The X values are a 64 digit string:\"<\/span>,<\/span> X<\/span>[<\/span>n<\/span>])<\/span>\r\nprint<\/span>(<\/span>\"and the y is the value of the number\"<\/span>,<\/span> Y<\/span>[<\/span>n<\/span>])<\/span>\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
    \n
    \n
    \n
    <\/div>\n
    \n
    The X values are a 64 digit string: [ 0.  0.  4. 10. 16. 16.  7.  0.  0.  3. 16. 13. 11. 16.  2.  0.  0.  1.\r\n  3.  0. 10.  9.  0.  0.  0.  0.  5.  8. 14. 15. 13.  0.  0.  0. 15. 16.\r\n 14. 12.  8.  0.  0.  0.  3. 12.  7.  0.  0.  0.  0.  0.  0. 15.  4.  0.\r\n  0.  0.  0.  0.  3. 14.  1.  0.  0.  0.]\r\nand the y is the value of the number 7\r\n<\/pre>\n<\/div>\n<\/div>\n
    \n
    <\/div>\n
    \"image\"<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
    \n
    \n
    In\u00a0[5]:<\/div>\n
    \n
    \n
    \n
    ## The first step is to break the data into the training set, and the test set.  <\/span>\r\n## Luckily, sklearn has you covered:<\/span>\r\n\r\nX_train<\/span>,<\/span> X_test<\/span>,<\/span> Y_train<\/span>,<\/span> Y_test<\/span> =<\/span> train_test_split<\/span>(<\/span>X<\/span>,<\/span>Y<\/span>,<\/span> test_size<\/span>=<\/span>0.2<\/span>)<\/span>\r\n\r\n## test_size tells the algrithm that we want to reserve 20% for the test (or validation) set.  Try it!  <\/span>\r\n## Now redo it with a test set at 30%<\/span>\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
    \n
    <\/div>\n
    \n
    \n
    Assignment Questions\u00b6<\/a><\/h2>\n
      \n
    1. How big is the full data set? How many numbers in training set? How many are held back in the test set? You should have 30% in your test set.<\/li>\n
    2. What is the L1 distance between X[15]<\/code> and X[25]<\/code>? (Manhattan)<\/li>\n
    3. What is the L2 distance between X[15]<\/code> and X[25]<\/code>? (Euclidean)<\/li>\n
    4. What is the L3 distance between X[15]<\/code> and X[25]<\/code>?<\/li>\n<\/ol>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      \n
      In\u00a0[1]:<\/div>\n
      \n
      \n
      \n
      ## Solutions<\/span>\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      \n
      In\u00a0[1]:<\/div>\n
      \n
      \n
      \n
      ## Lets do it!<\/span>\r\nfrom<\/span> sklearn.neighbors<\/span> import<\/span> *<\/span>\r\n\r\n## Run this to learn about the function:<\/span>\r\n?<\/span>KNeighborsClassifier\r\n\r\n## FYI: Minkowski distance means the Lp norm, so we can set p = 2 for Euclidean distance<\/span>\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      \n
      In\u00a0[7]:<\/div>\n
      \n
      \n
      \n
      ## For now, I'm going to set K = 3, and use the Euclidean or L2 norm.  <\/span>\r\n\r\n#Pick a name for your model -> I'm going to use a random word: cupcake<\/span>\r\n\r\n#step 1: let python know what algorithm I'm using:<\/span>\r\ncupcake<\/span> =<\/span>neighbors<\/span>.<\/span>KNeighborsClassifier<\/span>(<\/span>n_neighbors<\/span>=<\/span>15<\/span>,<\/span> p<\/span> =<\/span>2<\/span> )<\/span>\r\n#step 2: Train the model on ONLY THE TRAINING DATA (fit!)<\/span>\r\ncupcake<\/span>.<\/span>fit<\/span>(<\/span>X_train<\/span>,<\/span>Y_train<\/span>)<\/span>\r\n\r\n\r\n##Now we have a well trained model!  But is it a good model?<\/span>\r\n## I'm going to look at the errors in BOTH my training and test sets.  <\/span>\r\n## We do this by predicting what Y will be for each set, than actually comparing<\/span>\r\n\r\nY_pred_train<\/span> =<\/span> cupcake<\/span>.<\/span>predict<\/span>(<\/span>X_train<\/span>)<\/span> \r\nY_pred<\/span> =<\/span> cupcake<\/span>.<\/span>predict<\/span>(<\/span>X_test<\/span>)<\/span>\r\n\r\n##record errors<\/span>\r\nerror_train<\/span>=<\/span>list<\/span>(<\/span>Y_train<\/span> ==<\/span> Y_pred_train<\/span>)<\/span>.<\/span>count<\/span>(<\/span>False<\/span>)<\/span>\/<\/span>len<\/span>(<\/span>Y_train<\/span>)<\/span>\r\nerror_test<\/span>=<\/span>list<\/span>(<\/span>Y_test<\/span> ==<\/span> Y_pred<\/span>)<\/span>.<\/span>count<\/span>(<\/span>False<\/span>)<\/span>\/<\/span>len<\/span>(<\/span>Y_test<\/span>)<\/span>\r\n\r\n##Look it over:<\/span>\r\nprint<\/span>(<\/span>\"The training error was: \"<\/span>,<\/span> error_train<\/span>)<\/span>\r\nprint<\/span>(<\/span>\"The testing error was: \"<\/span>,<\/span> error_test<\/span>)<\/span>\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      \n
      \n
      <\/div>\n
      \n
      The training error was:  0.016701461377870562\r\nThe testing error was:  0.025\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      <\/div>\n
      \n
      \n
      Your turn:\u00b6<\/a><\/h2>\n
      For this week, we are going to turn a blind eye to the fact that we are experimenting on the test data. Try out different models! Look at different [latex]K[\/latex] values, and different [latex]p[\/latex] values to see if you can find something that works better than others. Can you automate this process using some loops or other tools?<\/p>\n
      Explain your best model, and hand it in. Expect that random change may affect different models.<\/p>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      \n
      In\u00a0[\u00a0]:<\/div>\n
      \n
      \n
      \n
       \r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"author":883,"menu_order":2,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[],"contributor":[],"license":[],"class_list":["post-61","chapter","type-chapter","status-publish","hentry"],"part":64,"_links":{"self":[{"href":"https:\/\/pressbooks.bccampus.ca\/businessanalytics\/wp-json\/pressbooks\/v2\/chapters\/61","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pressbooks.bccampus.ca\/businessanalytics\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/pressbooks.bccampus.ca\/businessanalytics\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/businessanalytics\/wp-json\/wp\/v2\/users\/883"}],"version-history":[{"count":5,"href":"https:\/\/pressbooks.bccampus.ca\/businessanalytics\/wp-json\/pressbooks\/v2\/chapters\/61\/revisions"}],"predecessor-version":[{"id":82,"href":"https:\/\/pressbooks.bccampus.ca\/businessanalytics\/wp-json\/pressbooks\/v2\/chapters\/61\/revisions\/82"}],"part":[{"href":"https:\/\/pressbooks.bccampus.ca\/businessanalytics\/wp-json\/pressbooks\/v2\/parts\/64"}],"metadata":[{"href":"https:\/\/pressbooks.bccampus.ca\/businessanalytics\/wp-json\/pressbooks\/v2\/chapters\/61\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/pressbooks.bccampus.ca\/businessanalytics\/wp-json\/wp\/v2\/media?parent=61"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/businessanalytics\/wp-json\/pressbooks\/v2\/chapter-type?post=61"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/businessanalytics\/wp-json\/wp\/v2\/contributor?post=61"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/pressbooks.bccampus.ca\/businessanalytics\/wp-json\/wp\/v2\/license?post=61"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}