Repository navigation
Exception: Could not find CSV dataset at "xyz.csv" after bootstrapping. #257
Description
Activity
When I simply place the CSV file in the root of the project, it works. But, "data/xyz.csv" doesn't.
Hi Partha Mishra (@gdprtcl), in order to work with your data you need to create and configure a dataset as it is described here https://github.com/microsoft/MLOpsPython/blob/master/docs/custom_model.md#configure-training-data
The CSV file gets picked up and the dataset is configured (checked in AML Workspace) when I place my file outside in the root of my Project structure, not when inside the "data" folder.
Eugene Fedorenko (@eedorenko)- Why should it be picked up from "data" folder? You are saying you specified "file_name = 'xyz.csv'" which means it's in the root folder.
- Originally CSV files in data folder are not used for training, they are used for the example showing how to test data quality. That's it.
- If there is no already configured dataset, it automatically generates diabetes.csv file, saves the file to the AML blob storage and configures a dataset for it.
- In order to work with your data (xyz.csv) you need to create and configure a dataset as it is described here https://github.com/microsoft/MLOpsPython/blob/master/docs/custom_model.md#configure-training-data
Reacted by Partha MishraEugene Fedorenko (@eedorenko) thanks for the inputs. I understand those.
The intent was to make it work when it's present in the folder. Obviously, we could provide just the name and it gets picked up from the root folder.
I had already tried with both file_name = 'xyz.csv' & file_name = 'data/xyz.csv'. Both didn't work.By this time, I made it work with a small code tweak for file_name in _build_train_pipeline.py
if (dataset_name not in aml_workspace.datasets): # This call creates an example CSV from sklearn sample data. If you # have already bootstrapped your project, you can comment this line # out and use your own CSV. #create_sample_data_csv() # Use a CSV to read in the data set. file_name = 'xyz.csv' file_name = os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir, "data", file_name)) print(file_name) if (not os.path.exists(file_name)): raise Exception("Could not find CSV dataset at \"%s\"....)Great that you made it working for you! However, if you are using your own data, this condition if (dataset_name not in aml_workspace.datasets): should return false for you, you are not supposed to use/modify code below it as you are not supposed to have data files in your code. Perhaps, we need to make comments on that in the code more clear and delete the data folder as it brings confusion.
Reacted by Partha Mishra and MyEugene Fedorenko (@eedorenko) Yes, exactly. That will definitely be better.
- linked a pull request that will close this issueUpdate docs and pipeline status badge #303
on Jul 2, 2020
Hi Team,
I tried to bootstrap the MLOps project to work with another Model. I put the CSV file inside the "data" folder and changed accordingly in customer_churn_build_train_pipeline.py
But I'm getting the Exception that it couldn't find the CSV file.
I have specified file_name = 'xyz.csv' and commented out the rest.