Skip to content

Completely avoid S3FS #119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bethac07 wants to merge 8 commits into from
Closed

Completely avoid S3FS #119

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
e7ba3b0
download pipeline and CSV, never use S3FS
bethac07 Nov 15, 2020
a16d25c
download pipeline and CSV, never use S3FS
bethac07 Nov 15, 2020
71b4959
don't list
bethac07 Nov 15, 2020
8159684
make folders
bethac07 Nov 15, 2020
a1c672e
can't use the input folder right now
bethac07 Nov 15, 2020
8c2ca70
logging
bethac07 Nov 15, 2020
a2de0e9
hell if I know
bethac07 Nov 15, 2020
95341e3
hell if I know
bethac07 Nov 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
logging
  • Loading branch information
@bethac07
bethac07 committed Nov 15, 2020
commit 8c2ca70925ead2c61c1c63f840c3ae53b8da254b
5 changes: 4 additions & 1 deletion worker/cp-worker.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,9 +230,12 @@ def runCellProfiler(message):
if not os.path.exists(os.path.split(local_csv_name)[0]):
os.makedirs(os.path.split(local_csv_name)[0])
csv_in = pandas.read_csv(new_csv_name)
csv_in=csv_in.astype('str')
csv_in.replace(DATA_ROOT,localIn,regex=True, inplace=True)
csv_in.to_csv(local_csv_name,index=False)
print('Wrote updated CSV')
printandlog('Wrote updated CSV',logger)
csvhead = csv_in.head()
printandlog(csvhead,logger)
newtag = True
else:
newtag = False
Expand Down