Python

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›229 »

Symbolic Butt: Mar 22, 2009; (_!_); Buglord

but you're actually using that variable! (this is the dumbest tangent)

# ? Apr 12, 2017 14:10

Adbot: ADBOT LOVES YOU

# ? Apr 16, 2024 08:27

Cingulate: Oct 23, 2012; by Fluffdaddy

For real though, how did you oldsters ever live with Python 2 and its leaking list comps?

# ? Apr 12, 2017 14:14

Space Kablooey: May 6, 2009

Cingulate posted:

For real though, how did you oldsters ever live with Python 2 and its leaking list comps?

You mean the leaking variable that is used on building the comps or you mean like a memory leak or something like that?

How to live with the former is to just redefine the name before reusing.

# ? Apr 12, 2017 14:24

Cingulate: Oct 23, 2012; by Fluffdaddy

HardDiskD posted:

You mean the leaking variable that is used on building the comps

Yeah, [_ for this_var_will_leak in in some_iter]

In other news, I may be literally the worst coder ITT, but I got really good teacher evaluations on my recent Python class and we'll try and apply for funding for a bigger project for teaching Python to students, and for using Python to teach science to students :v:

# ? Apr 12, 2017 14:27

huhu: Feb 24, 2006

I'm starting to run into more advanced packages that are requiring things like Microsoft Visual C++. I've also been reading about how some things are too slow in Python so other languages are used to deal with this. Is there an article/video that explains this all well?

# ? Apr 12, 2017 16:36

Dominoes: Sep 20, 2007

Broad topic. Many python modules use C code to improve speed. When installing them from source, (Including pip without a wheel) you'll need a compiler; Visual C++ is used in Win. Which package are you referring to? You can usually avoid this by using Conda, or a Chris Gohlke binary.

To speed up Python, you can use Cython, Numba, vectorized code, PyPy etc, on the bottlenecks. What specifically do you want to know?

# ? Apr 12, 2017 17:06

huhu: Feb 24, 2006

Dominoes posted:

Broad topic. Many python modules use C code to improve speed. When installing them from source, (Including pip without a wheel) you'll need a compiler; Visual C++ is used in Win. Which package are you referring to? You can usually avoid this by using Conda, or a Chris Gohlke binary.

To speed up Python, you can use Cython, Numba, vectorized code, PyPy etc, on the bottlenecks. What specifically do you want to know?

You've given me enough vocab I had to look up that I more or less understand what I need to know now.

# ? Apr 12, 2017 17:33

FAT32 SHAMER: Aug 16, 2012

I'm working on rewriting a jython library and the wanker who wrote this spaghetti should be drawn and quartered just for using jython :argh:

# ? Apr 12, 2017 17:39

huhu: Feb 24, 2006

I started developing with Sublime Text and then switched to PyCharm since I've been working mostly with Python. However, my new job has me focusing on Flask apps and PyCharm seems to be a bit lacking when it comes to dealing with HTML, CSS, and JS. I miss all the awesome plugins that come with Sublime Text as well. Am I missing something or should I be looking at something else to be developing Flask stuff?

# ? Apr 12, 2017 20:59

The Fool: Oct 16, 2003

huhu posted:

I started developing with Sublime Text and then switched to PyCharm since I've been working mostly with Python. However, my new job has me focusing on Flask apps and PyCharm seems to be a bit lacking when it comes to dealing with HTML, CSS, and JS. I miss all the awesome plugins that come with Sublime Text as well. Am I missing something or should I be looking at something else to be developing Flask stuff?

I feel like I might be saying this too often at this point, but have you considered VSCode accepted Christ as your Lord and Savior

https://code.visualstudio.com/docs/languages/python

# ? Apr 12, 2017 21:13

a witch: Jan 12, 2017

huhu posted:

I started developing with Sublime Text and then switched to PyCharm since I've been working mostly with Python. However, my new job has me focusing on Flask apps and PyCharm seems to be a bit lacking when it comes to dealing with HTML, CSS, and JS. I miss all the awesome plugins that come with Sublime Text as well. Am I missing something or should I be looking at something else to be developing Flask stuff?

If you have the cash, consider shelling out for the IntelliJ IDEA ultimate version. It includes all of the features in PyCharm (slightly behind pycharm releases), plus HTML, JavaScript, and pretty much everything else.

People also seem to love VS Code, and it's free.

# ? Apr 12, 2017 21:20

baka kaba: Jul 19, 2003; PLEASE ASK ME, THE SELF-PROFESSED NO #1 PAUL CATTERMOLE FAN IN THE SOMETHING AWFUL S-CLUB 7 MEGATHREAD, TO NAME A SINGLE SONG BY HIS EXCELLENT NU-METAL SIDE PROJECT, SKUA, AND IF I CAN'T PLEASE TELL ME TO
EAT SHIT

The Fool posted:

I feel like I might be saying this too often at this point, but have you considered VSCode accepted Christ as your Lord and Savior

https://code.visualstudio.com/docs/languages/python

Haven't used it but this REST client extension for it might be useful too

# ? Apr 12, 2017 21:25

FAT32 SHAMER: Aug 16, 2012

Jetbrains makes a lot of really nice stuff and I really like IntelliJ for Java and JavaScript. I found Pycharm to be frustrating to use because most small Python apps aren't organized the way Pycharm seems to try to organize them

# ? Apr 12, 2017 21:25

huhu: Feb 24, 2006

The Fool posted:

I feel like I might be saying this too often at this point, but have you considered VSCode accepted Christ as your Lord and Savior

https://code.visualstudio.com/docs/languages/python

This is amazing.

# ? Apr 12, 2017 21:33

Eela6: May 25, 2007; Shredded Hen

I mostly use visual studio code as well. I don't do any webdev so I can't comment on its use for that.

# ? Apr 12, 2017 21:36

Proteus Jones: Feb 28, 2013

When I see how far it's come in a relatively short amount of time, I keep wanting to give VisualCode a spin on macOS. But then I remember how addicted I am to the integrated console and debugging stuff in PyCharm.

# ? Apr 12, 2017 21:38

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

huhu posted:

I started developing with Sublime Text and then switched to PyCharm since I've been working mostly with Python. However, my new job has me focusing on Flask apps and PyCharm seems to be a bit lacking when it comes to dealing with HTML, CSS, and JS. I miss all the awesome plugins that come with Sublime Text as well. Am I missing something or should I be looking at something else to be developing Flask stuff?

What?

PyCharm is amazing for HTML/CSS/JS. It's basically a superset of Webstorm which is already widely liked for webdev stuff. I often use PyCharm for projects that are only HTML/CSS/JS.

Maybe ask about some specific things you're missing and I bet you'll find that most of them are supported.

# ? Apr 12, 2017 22:20

QuarkJets: Sep 8, 2008

Cingulate posted:

For real though, how did you oldsters ever live with Python 2 and its leaking list comps?

I'm not sure that it has ever mattered to me; I just kind of assumed that the last value of the comprehension variable would be available in the local scope and didn't realize that this was actually undesired behavior.

e: I still use Python2, because I can't choose what our production systems run. A surprising number of people are in this boat

QuarkJets fucked around with this message at 22:45 on Apr 12, 2017

# ? Apr 12, 2017 22:41

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

QuarkJets posted:

I'm not sure that it has ever mattered to me; I just kind of assumed that the last value of the comprehension variable would be available in the local scope and didn't realize that this was actually undesired behavior.

This is exactly right. It was just the way it was. It was never a problem.

# ? Apr 12, 2017 23:23

Space Kablooey: May 6, 2009

I'm not sure if I dislike comprehensions leaking, tbh.

Python code:

Python 3.5.2 (default, Nov 17 2016, 17:05:23) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> [x for x in range(10)]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined
>>> for y in range(10):
...  print(y, end=' ')
... 
0 1 2 3 4 5 6 7 8 9
>>> y
9

Python 2.7.12 (default, Jul  1 2016, 15:12:24) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> [x for x in range(10)]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> x
9
>>> for y in range(10):
...  print y,
... 
0 1 2 3 4 5 6 7 8 9
>>> y
9

The Py3 example feels inconsistent to me, in the sense that the comprehension doesn't leak while the for does, but that might just be because I'm more used to Py2.

Space Kablooey fucked around with this message at 14:16 on Apr 13, 2017

# ? Apr 13, 2017 14:13

Dominoes: Sep 20, 2007

Whoah - I always assumed they leaked, and set my vars up so it wouldn't be an issue. FYI, javascript uses a 'let' command that explicitly makes a variable nonleaky.

# ? Apr 13, 2017 14:22

Cingulate: Oct 23, 2012; by Fluffdaddy

code:

python -m "this" | grep honking

# ? Apr 13, 2017 14:22

huhu: Feb 24, 2006

Thermopyle posted:

What?

PyCharm is amazing for HTML/CSS/JS. It's basically a superset of Webstorm which is already widely liked for webdev stuff. I often use PyCharm for projects that are only HTML/CSS/JS.

Maybe ask about some specific things you're missing and I bet you'll find that most of them are supported.

VS just does everything I thought Pycharm would do right out of the box.

# ? Apr 13, 2017 17:32

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Right, I got that from your first post. I'm asking like what?

# ? Apr 13, 2017 18:17

Methanar: Sep 26, 2013; by the sex ghost

Regex question:

I have the following bit of code that returns some json. I want to change the output to be <whatever>.sub.example.com. Normally I would be able to do this in awk, but in this particular case it needs to be handled in python and I'm not as good at python as I thought. How could I replace example.com with sub.example.com for all the dictionary entries? The filterIP bit works so I was trying to do that again, but I'm having a hard time with it.

code:

def filterIP(fullList):
   regexIP = re.compile(r'\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}$')
   return filter(lambda i: not regexIP.search(i), fullList)

def filterSub(fullList2):
   regexSub = re.compile(r'example.com, sub.example.com')
   return filter(lambda i: regexSub.search(i), fullList2)

groups = {key : filterSub(filterIP(list(set(items)))) for (key, items) in groups.iteritems() }

print(self.json_format_dict(groups, pretty=True))

  "role_1": [
    "type-1.example.com",
    "type-12-sfsdf-453-2.example.com"
  ]

# ? Apr 14, 2017 22:04

vikingstrike: Sep 23, 2007; whats happening, captain

If that's substitution is all you need to do, can you not just use the replace() string method? Do you need to use regex?

# ? Apr 14, 2017 23:15

Methanar: Sep 26, 2013; by the sex ghost

I'm not actually working with strings. It's a list.

code:

def filterSub(fullList2):
     return re.sub(r'example\.com$', 'sub.example.com', fullList2)

groups = {key : filterSub(filterIP(list(set(items)))) for (key, items) in groups.iteritems() }

print(self.json_format_dict(groups, pretty=True))

Traceback (most recent call last):
  File "./chef_inventory.py.blah", line 209, in <module>
    main()
  File "./chef_inventory.py.blah", line 205, in main
    ci.execute()
  File "./chef_inventory.py.blah", line 196, in execute
    self.list_nodes()
  File "./chef_inventory.py.blah", line 181, in list_nodes
    groups = {key : filterSub(filterIP(list(set(items)))) for (key, items) in groups.iteritems() }
  File "./chef_inventory.py.blah", line 181, in <dictcomp>
    groups = {key : filterSub(filterIP(list(set(items)))) for (key, items) in groups.iteritems() }
  File "./chef_inventory.py.blah", line 179, in filterSub
    return re.sub(r'example\.com$', 'sub.example.com', fullList2)
  File "/usr/lib/python2.7/re.py", line 155, in sub
    return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or buffer

It's a pretty big one too and I read somewhere that when you need to apply a regex multiple times (assuming I need a regex) I should use re.compile. I don't NEED to use a regex, I just thought that was the right way of doing this sort of thing.

Methanar fucked around with this message at 23:48 on Apr 14, 2017

# ? Apr 14, 2017 23:45

chutwig: May 28, 2001; BURLAP SATCHEL OF CRACKERJACKS

Methanar posted:

I'm not actually working with strings. It's a list.

If you have a long list of strings, like

code:

["blah.example.com", "blorp.example.com", "burp.example.com"]

...then I'd probably just do

code:

[x.replace('example.com', 'sub.example.com') for x in your_list_of_strings]

...assuming you're not worried about where "example.com" appears and whether there might be multiple instances of it (i.e., you don't need to anchor to the right).

# ? Apr 15, 2017 00:18

breaks: May 12, 2001

If you do need to anchor it to the right your regexp itself is fine. Just modify filterSub and filterIP to take a string instead of a list then do this:

code:

def filter_all_this_shit(your_fucking_list):
   return [filterSub(some_fucked_up_domain) for some_fucked_up_domain in set(your_fucking_list) if filterIP(some_fucked_up_domain)]

Then your dict comprehension looks something like:

code:

groups = {key: filter_all_this_shit(domains) for key, domains in d.iteritems()}

You can iterate a set so no need to go list->set->list.

You could also use map or whatever else I guess, but personally in this kind of situation I almost always prefer giving a name to a comprehension like this. I think it's a lot more readable, assuming you aren't as big a fan of gratuitous profanity as I am.

# ? Apr 15, 2017 00:47

Tigren: Oct 3, 2003

chutwig posted:

If you have a long list of strings, like
code:
["blah.example.com", "blorp.example.com", "burp.example.com"]
...then I'd probably just do
code:
[x.replace('example.com', 'sub.example.com') for x in your_list_of_strings]
...assuming you're not worried about where "example.com" appears and whether there might be multiple instances of it (i.e., you don't need to anchor to the right).

Yep, that's where I'd go too.

code:

def filterIP(fullList):
   regexIP = re.compile(r'\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}$')
   return filter(lambda i: not regexIP.search(i), fullList)

groups = {key : [domain.replace('example.com', 'sub.example.com') for domain in filterIP(list(set(items)))] for (key, items) in groups.iteritems() }

# ? Apr 15, 2017 00:51

Methanar: Sep 26, 2013; by the sex ghost

The data is actually more structured like this. I gave the replace snippet a shot, but it didn't do anything.

code:

{ 
 "1": [
    "2.example.com",
    "2.example.com"
  ],
  "2": [
    "2.example.com"
  ],
  "3": [
    "3.example.com",
    "4.example.com"
  ]
}

Now what I'm thinking is the json output is actually a string (I think), so maybe I can do my string manipulation after writing to json, but that's not going so well either.

code:

 
        groups = {key : filterIP(list(set(items))) for (key, items) in groups.iteritems() }

        s = self.json_format_dict(groups, pretty=True)
#       print(s)

        def filterSub(fullList2):
        return re.sub(r"example.com$", "sub.example.com", fullList2)

        print(filterSub(s))

# ? Apr 15, 2017 00:59

QuarkJets: Sep 8, 2008

Methanar posted:

I'm not actually working with strings. It's a list.

A list of strings?

# ? Apr 15, 2017 01:00

QuarkJets: Sep 8, 2008

Methanar posted:

The data is actually more structured like this. I gave the replace snippet a shot, but it didn't do anything.
code:
{ 
 "1": [
    "2.example.com",
    "2.example.com"
  ],
  "2": [
    "2.example.com"
  ],
  "3": [
    "3.example.com",
    "4.example.com"
  ]
} 
Now what I'm thinking is the json output is actually a string (I think), so maybe I can do my string manipulation after writing to json, but that's not going so well either.
code:
 
        groups = {key : filterIP(list(set(items))) for (key, items) in groups.iteritems() }

        s = self.json_format_dict(groups, pretty=True)
#       print(s)

        def filterSub(fullList2):
        return re.sub(r"example.com$", "sub.example.com", fullList2)

        print(filterSub(s))

What you have is a dictionary where the keys are strings and the values are lists of strings, and what you want to do is do a replace in all of those lists of strings. Tables-breaking pseudocode:

Python code:

s_replaced = {key : [domain.replace('example.com', 'sub.example.com') for domain in val] for key, val in s.iteritems()}

It's preferrable to use string operations instead of regex whenever you can, there are far fewer pitfalls and it's usually simpler and faster

QuarkJets fucked around with this message at 01:09 on Apr 15, 2017

# ? Apr 15, 2017 01:05

vikingstrike: Sep 23, 2007; whats happening, captain

Methanar posted:

The data is actually more structured like this. I gave the replace snippet a shot, but it didn't do anything.
code:
{ 
 "1": [
    "2.example.com",
    "2.example.com"
  ],
  "2": [
    "2.example.com"
  ],
  "3": [
    "3.example.com",
    "4.example.com"
  ]
} 
Now what I'm thinking is the json output is actually a string (I think), so maybe I can do my string manipulation after writing to json, but that's not going so well either.
code:
 
        groups = {key : filterIP(list(set(items))) for (key, items) in groups.iteritems() }

        s = self.json_format_dict(groups, pretty=True)
#       print(s)

        def filterSub(fullList2):
        return re.sub(r"example.com$", "sub.example.com", fullList2)

        print(filterSub(s))

If you iterating over a dictionary you'll need to update the comprehension accordingly. If your use case really is just the simple replace, I think regex is overkill here and makes for messier code for no reason.

# ? Apr 15, 2017 01:06

Methanar: Sep 26, 2013; by the sex ghost

Got it working as it's supposed to. I was being dumb.

This is exactly what I was looking for. Thanks a lot. I learned a few things today.

code:

s_replaced = {key : [domain.replace('example.com', 'sub.example.com') for domain in val] for key, val in s.iteritems()}

# ? Apr 15, 2017 01:25

death cob for cutie: Dec 30, 2006; dwarves won't delve no more
too much splatting down on Zot:4

funny Star Wars parody posted:

Jetbrains makes a lot of really nice stuff and I really like IntelliJ for Java and JavaScript. I found Pycharm to be frustrating to use because most small Python apps aren't organized the way Pycharm seems to try to organize them

Personally, I write all my Java in IntelliJ but only larger Python projects in PyCharm - if it's something small I'm A-OK with not having all the fancy features.

# ? Apr 15, 2017 21:08

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

funny Star Wars parody posted:

Jetbrains makes a lot of really nice stuff and I really like IntelliJ for Java and JavaScript. I found Pycharm to be frustrating to use because most small Python apps aren't organized the way Pycharm seems to try to organize them

What does this mean exactly? AFAIK, PyCharm doesn't care how your projects are organized.

I often go through a lot of different structures on a project. It starts out small with a few files just right next to each other, them I move stuff into packages then I move stuff into a full-blown thjngamajig with a setup.py, etc...

PyCharm doesn't seem to give a crap and in fact helps with that since it does a fine job with automatic refactoring.

# ? Apr 15, 2017 23:03

Dominoes: Sep 20, 2007

PyCharm doesn't like to work with files unless they're in a folder that includes PyCharm meta files (.idea directory). Maybe that's it?

# ? Apr 15, 2017 23:07

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

https://github.com/python/cpython/blob/3.6/Lib/collections/__init__.py

code:


    def __init__(*args, **kwds):

        [snip]

        self, *args = args

Is there a reason ever to write code this way like the init method of OrderedDict? Why wouldn't you explicitly write out "self" as the first argument?

# ? Apr 15, 2017 23:22

Adbot: ADBOT LOVES YOU

# ? Apr 16, 2024 08:27

Eela6: May 25, 2007; Shredded Hen

Boris Galerkin posted:

https://github.com/python/cpython/blob/3.6/Lib/collections/__init__.py
code:
    def __init__(*args, **kwds):

        [snip]

        self, *args = args
Is there a reason ever to write code this way like the init method of OrderedDict? Why wouldn't you explicitly write out "self" as the first argument?

I have no idea. This is really strange.

# ? Apr 15, 2017 23:40

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python

«‹›229 »