python - How do I tell a .py script to look for folders and file in the directory it is in?

Question

I have a .py script that pulls in data from Google Sheets and outputs it in a yaml format. This is for my Hugo powered website which is served via Netlify. As I understand it, Netlify is capable of running Python too so I thought I could upload the web content and the python file in the same directory. This is required for updating the content dynamically, and I expect the python file to run everytime I trigger a build for the website. However, the python file requires certain credentials to work.

My code currently looks like this:

# Set location to write new files to.
outputpath = Path("D:/content/submission/")

#Read JSON: 
json = Path("D:/credentials.json")

These are hardcoded local paths. When I bundle the script in the website directory, what paths should I write in so that when the script runs, these files are read in and outputted correctly?

I would want to output in my content/submission folder and read in from my creds/credentials.json. Should I just put these paths in? Will it know that it has to look within the directory for these folders, or is there something I need to add to the script that tells it to work within the directory it is sitting in?

Answer 1

?? First, credentials and secrets are best kept out of files (and esp, esp, source control).

For general file locations however, you can use something like:

pa_same_but_json = Path(__file__).with_suffix(".json")
pa_same_directory = Path(__file__).parent / "nosecrets.json")

To answer your comments:

(Mind you not 100% sure about Window Drives in the following):

parent

is an attribute on Path objects allowing you to "climb up" hierarchy.

Path("c:empfoo.json").parent returns same as Path("c:emp")

Yes, you can do mypath.parent.parent

/ is a path concatenation operator

when applied to Path objects

So

myfile = os.path.join(["c:", "temp", "foo.json"])

and

myfile_as_a_Path = Path("c:") / "temp" / "foo.json"

are the same, except for one being a string, the other a Path instance. Once the first Path has been built (on C:) the rest of the code "knows that it operating on Path instances" and re-purposes the division operator support (probably some magic __div__ method intended for instance math ) to support path concatenation. This happens because most operations on Path instances return another Path, allowing you to do this type of chaining.

It's best not to write way up the hierarchy in a hosted/VM context (you never know directory structure above or if you have permissions), but something based on your script location might be

pa_current = Path(__file__).parent

# could `content/submission` but that's assuming you're always on Posix
# systems.  Letting Pathlib do the work is safer, even if Windows probably
# puts up with `/`
pa_write = pa_current / "content" / "submission" 

pa_read = pa_current / "credentials.json"

These at this points are Path instances, but really not much different than strings except having smarter methods to manipulate them. They don't know or care if the files exist or not.

P.S.

?? A consideration is that, in many web contexts, writing to code directories (like what happens in a content/submission under the python scripts) is a security goof as well.

Maybe pa_write = pa_current.parent.parent / "uploads" / "content" / "submission" would be better.

Specifically when it comes to user uploads and secrets, please refer to best practices for your platform, not just what Python can do. This answer was about pathlib.Path, not Hugo uploads.