|
Dominoes posted:Naive: Based on the docs' "f.write(string) writes the contents of string to the file" and not seeing any examples like your naive one, I assumed incorrectly that each call to write would overwrite the entire file.
|
#
?
Jul 27, 2020 01:20
|
|
|
|
| # ? Apr 26, 2024 20:31 |
|
|
Dominoes posted:That's the one Googs was sending me to, but it's a tutorial. I'm looking for a breakdown of each function/method available. Their args, what they return, and a description. Eg so you could see that readlines returns a list of strings where it splits by /n char, loads the whole thing into mem at once etc. readline returns a single string of the next line in the stream if it's there, and raises (what?) if you're at the end etc.
|
|
#
?
Jul 27, 2020 01:29
|
|
|
That's it! Nailed it. Btw, the with statements in my example are set up so that even if something goes wrong with the Python code, the memory's freed and the file closes upon completion. Dominoes fucked around with this message at 01:41 on Jul 27, 2020 |
|
#
?
Jul 27, 2020 01:37
|
|
|
Dominoes posted:That's the one Googs was sending me to, but it's a tutorial. I'm looking for a breakdown of each function/method available. Their args, what they return, and a description. Eg so you could see that readlines returns a list of strings where it splits by /n char, loads the whole thing into mem at once etc. readline returns a single string of the next line in the stream if it's there, and raises (what?) if you're at the end etc. With open... As name and for line in name works fine.
|
|
#
?
Jul 27, 2020 01:39
|
|
|
sed -u -i 's/stringtoreplace/replacement/g' file.txt Malcolm XML fucked around with this message at 16:39 on Jul 27, 2020 |
|
#
?
Jul 27, 2020 16:31
|
|
|
Bitching: developing on windows, particularly when I’ll have to deploy to Amazon Linux, is loving dumb and I’m dumb for doing it Guess who spent half the day trying to figure out why I couldn’t get something to work only to find out it was because I was making a tarball that had different folder slashes than the deployed machine? This idiot. Don’t be like me.
|
|
#
?
Jul 29, 2020 02:40
|
|
|
I recommend WSL if you're going to make tarballs. It's built into most editions as a CLI tool. The command's something I have to google and C+P every time.
|
|
#
?
Jul 29, 2020 03:00
|
|
|
Dominoes posted:I recommend WSL if you're going to make tarballs. It's built into most editions as a CLI tool. The command's something I have to google and C+P every time. Yea I have the Ubuntu WSL installed already and even have anaconda, AWS cli, etc working on it from the last time I didn’t learn my lesson because I’m the terrible programming I’m just so used to my pycharm workflow and for most stuff it doesn’t make a difference. Edit I should see if windows pycharm can somehow use the WSL Edit2 Of course it can god drat jetbrains you’re too good https://www.jetbrains.com/help/pycharm/using-wsl-as-a-remote-interpreter.html CarForumPoster fucked around with this message at 03:12 on Jul 29, 2020 |
|
#
?
Jul 29, 2020 03:05
|
|
|
I'm using a def. Is there a way to define, such that the editor displays a message to the user when they try to invoke it (like they start typing the function and a little autocomplete bubble appears with the message about the function, like it's inputs). I've seen it done for classes, similar to:code:code:Is this A Thing? PRADA SLUT fucked around with this message at 04:47 on Aug 4, 2020 |
|
#
?
Aug 4, 2020 04:45
|
|
|
If I understand you, that's a function of your IDE. PyCharm is by far the best at that sort of thing. Using typehints helps.
|
|
#
?
Aug 4, 2020 06:46
|
|
|
Dominoes posted:Is this A Thing? As Dominoes said, the type information for arguments/returns is best covered by type hints (which PyCharm has robust support for, like seemingly everything else). If you are dynamically adding methods to a class or something you can add type hints by using a stub. The other stuff people put in multi-line comments at the top of classes/functions almost always seems unnecessary to me. It's either high level information that can be conveyed by just naming the function appropriately, or its specific enough that I'd usually rather it was in regular comments where it's relevant. Maybe it's just me, but who is this poo poo for? Python code:Wallet fucked around with this message at 15:02 on Aug 4, 2020 |
|
#
?
Aug 4, 2020 14:57
|
|
|
Wallet posted:As Dominoes said, the type information for arguments/returns is best covered by type hints (which PyCharm has robust support for, like seemingly everything else). If you are dynamically adding methods to a class or something you can add type hints by using a stub. The other stuff people put in multi-line comments at the top of classes/functions almost always seems unnecessary to me. It's either high level information that can be conveyed by just naming the function appropriately, or its specific enough that I'd usually rather it was in regular comments where it's relevant. People generating documentation with something like sphinx for software they intend others to use.
|
|
#
?
Aug 4, 2020 17:28
|
|
|
Wallet posted:As Dominoes said, the type information for arguments/returns is best covered by type hints (which PyCharm has robust support for, like seemingly everything else). If you are dynamically adding methods to a class or something you can add type hints by using a stub. The other stuff people put in multi-line comments at the top of classes/functions almost always seems unnecessary to me. It's either high level information that can be conveyed by just naming the function appropriately, or its specific enough that I'd usually rather it was in regular comments where it's relevant. What Bundy said, but also that is indeed a somewhat useless docstring. It doesn't describe anything that the function name doesn't cover. But what if you start making it more in depth so that devs don't have to read the function's code and follow its dependencies, or can view it on that online documentation? Like, Python code:
|
|
#
?
Aug 4, 2020 17:44
|
|
|
Dominoes posted:If I understand you, that's a function of your IDE. PyCharm is by far the best at that sort of thing. Using typehints helps. Yeah, I'm on VSCode and it does that as well. I thought maybe there was some standard way of formatting that IDE's would read.
|
|
#
?
Aug 4, 2020 19:04
|
|
|
There are several standard formats, look up "docstring formatting". My favorite is numpy style. That said, the IDE popup just shows the whole thing so for your purposes it doesn't matter. BTW, for one of the posters above, docstrings aren't comments! It's a pet peeve of mine. If you use the triple quotes syntax as a comment you're really just creating a string but not assigning it to any variable.
|
|
#
?
Aug 4, 2020 19:15
|
|
|
Bundy posted:People generating documentation with something like sphinx for software they intend others to use. Yeah, I know, but all that accomplishes is an automatically generated web page that tells me that your function does the thing that the name has already told me it does. If you want to put actually useful information in there I get it (even if I don't particularly like it) but maybe 1/10 projects I look at that use docstrings have anything even vaguely useful contained in them, and in the cases where they do have useful information there's often so much bloat that the file is 75% docstring and 25% code. SurgicalOntologist posted:BTW, for one of the posters above, docstrings aren't comments! It's a pet peeve of mine. If you use the triple quotes syntax as a comment you're really just creating a string but not assigning it to any variable. Fair enough—I don't personally use that syntax unless I need a multi-line string and I'm feeling lazy about line breaks, but I see people use them as comments all the time instead of adding a few extra pound signs. Wallet fucked around with this message at 23:02 on Aug 4, 2020 |
|
#
?
Aug 4, 2020 22:57
|
|
|
Please document the assumptions your code makes and its semantics. Please don't say "just read the code". Especially if you're not using typing (with good, well-named, verbose, and specific types) it is really important to keeping the code readable.
|
|
#
?
Aug 4, 2020 23:03
|
|
|
Phobeste posted:Please document the assumptions your code makes and its semantics. Please don't say "just read the code". Especially if you're not using typing (with good, well-named, verbose, and specific types) it is really important to keeping the code readable. I'd rather docstrings than nothing, but I'd rather good type hints, comments, and if necessary external documentation than docstrings 
|
|
#
?
Aug 4, 2020 23:36
|
|
|
Question on a "makefile" for Python. I'm using a Docker 3.8 Slim Buster python environment, and I have a few libraries in a requirements.txt For Reasons, I need a makefile that installs all the required packages that the project uses onto the base system, both from the 3.8 environment. How would I check what libraries the Docker environment uses?
|
|
#
?
Aug 18, 2020 17:40
|
|
|
Generate your requirements.txt file via pip freeze. Itwill output a full list of installed packages, including what version they are.
|
|
#
?
Aug 18, 2020 18:29
|
|
|
Master_Odin posted:Generate your requirements.txt file via pip freeze. Itwill output a full list of installed packages, including what version they are. Do you guys really do it this way for deployed apps? I find over-specifying requirements leads to a hell-scape of conflicts 6 months later and instead will update my requirements.txt as I pip install things on a new project with the specific versions of the packages I want but not specific versions of the dependencies.
|
|
#
?
Aug 18, 2020 21:27
|
|
|
Yeah ideally you want your reqs to be as slim as possible, just adding the non-dependencies as you build up the app, and no dependencies. But depending on the OP's Reasons, starting from a pip freeze will get it working, and then weeding out the dependencies can come later. :techdebt:
|
|
#
?
Aug 18, 2020 21:40
|
|
|
Isn't something like pipenv the preferred way now, anyhow? In that you track the specific requirements YOUR code is bringing, and it'll let you operate the requirements at that level, and deal with the lower level details separately. Still ultimately working via requirements.txt (or not?) but not in the raw "your requirements and everything your requirements require and everything your requirements' requirements require and so on, all intermingled" way that pip freeze does.
|
|
#
?
Aug 18, 2020 22:14
|
|
|
Oh I was speaking under the assumption that we’re doing all of it in some form of pipenv/venv. No way am I gonna do any of that poo poo with global python.
|
|
#
?
Aug 18, 2020 23:05
|
|
|
i am new to python so this is probably pretty easy. let's say i have an if/else loop. if condition x, i want to set variable a = 'stringA', b = intB, and c = 'stringC'. if not condition x, i want to set variable a = 'stringAmod', b = intBmod, and c = 'stringCmod'. that's easy enough. what i'm unclear on is how best to combine the entire result set a, b, and c from the first condition to the entire result set a, b, and c from the else condition, and then write that entire set to a csv. now, there's a simple way--in each condition, write each row to a csv after defining the variables. something like: if x is true: a = stringA b = intB c = stringC write.writerow(a, b, c) else: a = stringAmod b = intBmod c = stringCmod write.writerow(a, b, c) -- however, i feel like a more efficient way would be to do something like: if x is true: a = stringA b = intB c = stringC #i have no idea what the right thing here is, hence my question jsonlikeobject = add(a, b, c) else: a = stringAmod b = intBmod c = stringCmod jsonlikeobject = add(a, b, c) writerow(cycle through jsonlikeobject, inserting each 'array' or whatever you'd call it. does this make sense? if so, what am i looking for? thanks. abelwingnut fucked around with this message at 23:24 on Aug 18, 2020 |
|
#
?
Aug 18, 2020 23:21
|
|
|
Keep it as simple as you need for your use. Your pseudocode is fine.Python code:
|
|
#
?
Aug 18, 2020 23:29
|
|
|
Master_Odin posted:Generate your requirements.txt file via pip freeze. Itwill output a full list of installed packages, including what version they are. How do you get this when you use a Docker environment? Also, does this show you only the files the project uses?
|
|
#
?
Aug 19, 2020 07:48
|
|
|
I'm trying to share some state (less than 1 KB of tabular data, could easily be a CSV or list of dicts) between two types of Python processes running on the same system. This would be one-to-many... one process writes the state, a handful of separate ones will read it. Any ideas for how to sync between them? I don't control spawning the reader processes and they aren't run by the same user, so it exceeds my normal use of pipes/queues. I could probably use a file but how to ensure the file is flushed and fully written to disk for the helper to read? I guess I could use a specific file terminator and only have the reader update internal copy of state when it sees that. Similar file access and concurrency issues between sqlite or something. This seems like a problem that is reasonably common yet in a spot where standing up Redis server seems like overkill. Does anyone have any pointers in a specific direction?
|
|
#
?
Aug 21, 2020 03:40
|
|
|
A naive solution might to be writing a tmp file and then moving it to the correct location, as long as you aren't required to append to the same file over time that should work.
|
|
#
?
Aug 21, 2020 05:55
|
|
|
Hed posted:I'm trying to share some state (less than 1 KB of tabular data, could easily be a CSV or list of dicts) between two types of Python processes running on the same system. This would be one-to-many... one process writes the state, a handful of separate ones will read it. Any ideas for how to sync between them? I don't control spawning the reader processes and they aren't run by the same user, so it exceeds my normal use of pipes/queues. I could probably use a file but how to ensure the file is flushed and fully written to disk for the helper to read? I guess I could use a specific file terminator and only have the reader update internal copy of state when it sees that. Similar file access and concurrency issues between sqlite or something. I'd just use mmap for this - memory mapped files generally work well for this use case.
|
|
#
?
Aug 21, 2020 07:00
|
|
|
necrotic posted:A naive solution might to be writing a tmp file and then moving it to the correct location, as long as you aren't required to append to the same file over time that should work. If there are sync issues and you need to avoid writing the file while another process may be reading it a lock file would also work, just create an adjacent file on disk and any process that might read from the main file is expected to block as long as that lock file is present. Create the lock file, write the data (or replace the file with the tmp file as suggested), remove the lock file. Of course, maybe someone is reading the data file when you create the lock file, so maybe you need a little pause in there after creating the lock for any actively ongoing reads.
|
|
#
?
Aug 21, 2020 16:36
|
|
|
Folks is there an idiot-proof way to handle and manage pip packages? I'm forever seeming to have just every drat thing break when trying to set up some interesting looking app or the other and it's continuously a pain in my side.
|
|
#
?
Aug 21, 2020 16:39
|
|
|
Warbird posted:Folks is there an idiot-proof way to handle and manage pip packages? I'm forever seeming to have just every drat thing break when trying to set up some interesting looking app or the other and it's continuously a pain in my side. On the publisher side, nothing special. On the user side, use virtualenvs. You create a virtualenv for each "app". You can activate or deactivate them. When one is activated, the shell that it's activated in will use the virtualenv's configuration to find and install python packages. You should do this even if you have just one app you need to install, honestly - it makes life so much better.
|
|
#
?
Aug 21, 2020 16:45
|
|
|
Good to know. I’ll keep that in mind when I reinstall my OS because I think I just royally hosed up everything trying to “fix” the problem.
|
|
#
?
Aug 21, 2020 17:18
|
|
|
Virtualenvs are definitely not idiot-proof compared to conda but do have plenty of advantages.
|
|
#
?
Aug 22, 2020 00:30
|
|
|
Well it appears my mobo/bios is utterly hosed so I guess I don’t have to sweat it.
|
|
#
?
Aug 22, 2020 00:35
|
|
|
Warbird posted:Well it appears my mobo/bios is utterly hosed so I guess I don’t have to sweat it. A morality tale for messing around with OS-level pip
|
|
#
?
Aug 22, 2020 00:55
|
|
|
Anyone interested in a small freelance gig? A year or two ago I helped someone set up a script that scrapes some sports betting websites and runs some stats through his formula and spits out the good bets or whatever. He wants to update it because of some changes to the sites probably. It's probably only a few hours of work but I don't have the energy these days with the startup and the baby. My code kind of sucks -- I was really phoning it in -- but if you can handle requests, beautifulsoup, and pandas it won't be too difficult. And who knows, maybe the bets are valuable (I never tried). Anyways, PM me if you're interested and I'll put you in touch.
|
|
#
?
Aug 22, 2020 16:06
|
|
|
SurgicalOntologist posted:Anyone interested in a small freelance gig? A year or two ago I helped someone set up a script that scrapes some sports betting websites and runs some stats through his formula and spits out the good bets or whatever. He wants to update it because of some changes to the sites probably. It's probably only a few hours of work but I don't have the energy these days with the startup and the baby. Shot you a PM.
|
|
#
?
Aug 23, 2020 01:49
|
|
|
|
| # ? Apr 26, 2024 20:31 |
|
|
TLDR: How can I figure out the minimum number of packages needed in a venv/my environment? Background: I'm making several AWS lambda functions that I'm deploying with Zappa with a lot of piping and testing different ways to do it. For example wrapping them with a Flask API, making them part of my Django site, embedding different add ons, etc. I just hit the AWS Lambda file size limit and Zappa's slim_handler seems to be busted. (Zip file is 111MB, unzipped its north of 200MB) I think its bloated for two reasons: -My venv has gotten quite bloated mostly with dependencies I don't need when I pip installed something that I later uninstalled. -Packages like pandas do a ton but I'm only using a tiny subset of their functionality. Is the only thing I can do to delete my venv and rerun my requirements.txt? Is there any way to not include some dependencies? EDIT: Deleted venv and reinitialized. The venv folder that pycharm creates is 75MB with only pip20 and setuptools...is that normal? Also my unzipped file is still >250MB. Looks like Ill either need to install less, get slim handler working...or...something CarForumPoster fucked around with this message at 12:51 on Aug 26, 2020 |
|
#
?
Aug 26, 2020 12:10
|
|