Read csv into pandas from disk?

Technical Topics PyScript
1

Hi all,
Can I read a csv from disk into pandas? When I hard-link to the file, it says it can’t find the file. If I do os.getcwd(), I get /home/pyodide, which does not exist. All the examples I can find on the website either simulate data or grab it from the web.

Here is my code:

<html>
  <head>
    <link rel="stylesheet" href="https://pyscript.net/alpha/pyscript.css" />
    <script defer src="https://pyscript.net/alpha/pyscript.js"></script>
  </head>
  <body> 
    <py-env>
    - numpy
    - pandas
  </py-env>
  
  <py-script>import numpy as np
import pandas as pd
import os
print(os.getcwd())
df = pd.read_csv('/home/david/pyscript-main/pyscriptjs/public/voice_data.csv')
print(df.head())

Here is the output:

/home/pyodide
JsException(PythonError: Traceback (most recent call last): File "/lib/python3.10/site-packages/_pyodide/_base.py", line 429, in eval_code .run(globals, locals) File "/lib/python3.10/site-packages/_pyodide/_base.py", line 300, in run coroutine = eval(self.code, globals, locals) File "", line 5, in File "/lib/python3.10/site-packages/pandas/util/_decorators.py", line 311, in wrapper return func(*args, **kwargs) File "/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 680, in read_csv return _read(filepath_or_buffer, kwds) File "/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 575, in _read parser = TextFileReader(filepath_or_buffer, **kwds) File "/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 933, in __init__ self._engine = self._make_engine(f, self.engine) File "/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1217, in _make_engine self.handles = get_handle( # type: ignore[call-overload] File "/lib/python3.10/site-packages/pandas/io/common.py", line 789, in get_handle handle = open( FileNotFoundError: [Errno 44] No such file or directory: '/home/david/pyscript-main/pyscriptjs/public/voice_data.csv' )
2 Likes
2

Same issue with me. The CSV file is placed in the same directory as that of the HTML file and it cannot be handled by PyScript? Is there any particular place we need to place the CSV file in? I get the File Not Found error as well.

1 Like
3

The python code (and all browser runtime) runs in a sandbox with a virtual or “fake” filesystem, not the real local filesystem. This is a safety feature of browsers, to stop downloaded scripts from being able to read your local data. None of the os module’s features will behave as you expect them to.

If you are hosting your own CSV file in a directory with the HTML, you can probably do an HTTP fetch, as in the examples, but with a filepath like “./voice_data.csv” - i.e., relative to the location of the HTML. It the CSV file is remote, it needs to be on a server set up to allow CORS from any location.

1 Like
4

you can try input file selector. once file is detected it can be loaded from the selected location into the browser.

5

For me in order to read from the csv file, i placed it in the same directory as my html file and in the py-env tag i placed:

- paths:
` - /fileName.csv

then you can call it in the py-script like so:

import pandas as pd
df = pd.read_csv("./fileName.csv")
df

PS: Don’t forget to start the local server

6

Did exactly what you have said here but it is not working for me…

<py-env>
  - altair
  - pandas
  - vega_datasets
  - paths:
` - /weatherAUS.csv
</py-env>

…

within

df = pd.read_csv("./weatherAUS.csv")

I am getting error now
JsException(PythonError: Traceback (most recent call last): File “/lib/python3.10/site-packages/_pyodide/_base.py”, line 429, in eval_code .run(globals, locals) File “/lib/python3.10/site-packages/_pyodide/_base.py”, line 300, in run coroutine = eval(self.code, globals, locals) File “”, line 1, in ModuleNotFoundError: No module named ‘altair’ )

If I remove path line and df lines from the script code is working… Basically, I took working example of altair.html and introducing reading data from local csv file…

7

I think for the paths, you forgot the indentation for the file name it should be

  • paths:
    • filename.csv
8 
<py-env>
  - altair
  - pandas
  - vega_datasets
  - paths:
    - ./weatherAUS.csv
</py-env>

That should work, worked for me. Make sure to include the “./” before the filepath and make sure that line has a second indent.

10

You try to read from webasm virtual fs (File System API — Emscripten 3.1.9-git (dev) documentation).
For os filesystem files you must <py-env> or fetch()

11

When you specify the file name “filename.ext” while read file, you are providing the open() function with a relative path. This means that the file you want to read is located in the current working directory.

file = open('filename.ext') //relative path

In the above code, you are not giving the full path to a file to the open() function, just its name - a relative path. The error “FileNotFoundError: [Errno 2] No such file or directory” is telling you that there is no file of that name in the working directory. So, try using the exact, or absolute path.

file = open(r'C:\path\to\your\filename.ext') //absolute path

In the above code, all of the information needed to locate the file is contained in the path string - absolute path.

If the full path to the file is not provided, the python file path is interpreted relative to the current working directory. This is the directory from which the program was started. For this to work, the directory containing the python executable must be in the PATH environment variable, which contains directories automatically searched for executables when a command is entered. To access a file in a different directory, you must either specify a relative path between the files or use an absolute path for one of them.

12

Hi @carlhyde - while your answer is correct (and excellent!) for general desktop Python, it is not accurate for PyScript, which does not have an internal concept of a PATH variable or Python Executable in same way that “normal” python does. Files from the desktop/server environment need to be moved into the Emscripten virtual filesystem in the browser and accessed from there - using a local file directly like the original poster did will not work.

Since this is quite an old thread, I should note that the syntax for brining files from a remote URL into the virtual filesystem changed in PyScript 2022.11.1 to use [[fetch]] configurations in the <py-config> tag, as opposed to paths in <py-env>.

13

Once relative import is used (in this case importing .csv from the root folder), is it then no longer possible to just share the .html so that the receiver can just open it and see the result ? (And hence, a static web server is required ? )
If so, is there a dirty way of embedding the data into html ?