Read csv into pandas from disk?

Technical Topics PyScript
#1

Hi all,
Can I read a csv from disk into pandas? When I hard-link to the file, it says it can’t find the file. If I do os.getcwd(), I get /home/pyodide, which does not exist. All the examples I can find on the website either simulate data or grab it from the web.

Here is my code:

<html>
  <head>
    <link rel="stylesheet" href="https://pyscript.net/alpha/pyscript.css" />
    <script defer src="https://pyscript.net/alpha/pyscript.js"></script>
  </head>
  <body> 
    <py-env>
    - numpy
    - pandas
  </py-env>
  
  <py-script>import numpy as np
import pandas as pd
import os
print(os.getcwd())
df = pd.read_csv('/home/david/pyscript-main/pyscriptjs/public/voice_data.csv')
print(df.head())

Here is the output:

/home/pyodide
JsException(PythonError: Traceback (most recent call last): File "/lib/python3.10/site-packages/_pyodide/_base.py", line 429, in eval_code .run(globals, locals) File "/lib/python3.10/site-packages/_pyodide/_base.py", line 300, in run coroutine = eval(self.code, globals, locals) File "", line 5, in File "/lib/python3.10/site-packages/pandas/util/_decorators.py", line 311, in wrapper return func(*args, **kwargs) File "/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 680, in read_csv return _read(filepath_or_buffer, kwds) File "/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 575, in _read parser = TextFileReader(filepath_or_buffer, **kwds) File "/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 933, in __init__ self._engine = self._make_engine(f, self.engine) File "/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1217, in _make_engine self.handles = get_handle( # type: ignore[call-overload] File "/lib/python3.10/site-packages/pandas/io/common.py", line 789, in get_handle handle = open( FileNotFoundError: [Errno 44] No such file or directory: '/home/david/pyscript-main/pyscriptjs/public/voice_data.csv' )
4 Likes
#2

Same issue with me. The CSV file is placed in the same directory as that of the HTML file and it cannot be handled by PyScript? Is there any particular place we need to place the CSV file in? I get the File Not Found error as well.

1 Like
#3

The python code (and all browser runtime) runs in a sandbox with a virtual or “fake” filesystem, not the real local filesystem. This is a safety feature of browsers, to stop downloaded scripts from being able to read your local data. None of the os module’s features will behave as you expect them to.

If you are hosting your own CSV file in a directory with the HTML, you can probably do an HTTP fetch, as in the examples, but with a filepath like “./voice_data.csv” - i.e., relative to the location of the HTML. It the CSV file is remote, it needs to be on a server set up to allow CORS from any location.

2 Likes
#4

you can try input file selector. once file is detected it can be loaded from the selected location into the browser.

#5

For me in order to read from the csv file, i placed it in the same directory as my html file and in the py-env tag i placed:

- paths:
` - /fileName.csv

then you can call it in the py-script like so:

import pandas as pd
df = pd.read_csv("./fileName.csv")
df

PS: Don’t forget to start the local server

1 Like
#6

Did exactly what you have said here but it is not working for me…

<py-env>
  - altair
  - pandas
  - vega_datasets
  - paths:
` - /weatherAUS.csv
</py-env>

…

within

df = pd.read_csv("./weatherAUS.csv")

I am getting error now
JsException(PythonError: Traceback (most recent call last): File “/lib/python3.10/site-packages/_pyodide/_base.py”, line 429, in eval_code .run(globals, locals) File “/lib/python3.10/site-packages/_pyodide/_base.py”, line 300, in run coroutine = eval(self.code, globals, locals) File “”, line 1, in ModuleNotFoundError: No module named ‘altair’ )

If I remove path line and df lines from the script code is working… Basically, I took working example of altair.html and introducing reading data from local csv file…

#7

I think for the paths, you forgot the indentation for the file name it should be

  • paths:
    • filename.csv
1 Like
#8 
<py-env>
  - altair
  - pandas
  - vega_datasets
  - paths:
    - ./weatherAUS.csv
</py-env>

That should work, worked for me. Make sure to include the “./” before the filepath and make sure that line has a second indent.

2 Likes
#10

You try to read from webasm virtual fs (File System API — Emscripten 3.1.9-git (dev) documentation).
For os filesystem files you must <py-env> or fetch()