Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Cingulate: Oct 23, 2012; by Fluffdaddy

Also note that Pandas has some basic plotting capacities (MPL-based ofc).

# ? May 21, 2015 21:03

Adbot: ADBOT LOVES YOU

# ? Jun 3, 2024 16:18

nonathlon: Jul 9, 2004; And yet, somehow, now it's my fault ...

OnceIWasAnOstrich posted:

I notice your requirements didn't include being Pythonic... There is always the option of plotting by programming R in Python with ggplot. The interface is rather R-like, but ggplot in R makes a lot more sense than Matlab's plotting so it is a step up, and it is integrated fairly well with Pandas and statsmodels,

Call that an implied requirement. My main beef with matplotlib (while admitting it's a powerful package lots of people love) is that it just doesn't stick to my brain and feels like I'm swapping to another language when I do plotting. And should you look at the source to see how things work, you're confronted with a whole lot of functions using kwargs.

Doing an R call may be the way to go.

# ? May 22, 2015 14:16

OnceIWasAnOstrich: Jul 22, 2006

To be clear, the library I linked is actually a Python library that just happens to have an interface very very similar to ggplot has in R.

# ? May 22, 2015 14:55

taqueso: Mar 8, 2004

OnceIWasAnOstrich posted:

To be clear, the library I linked is actually a Python library that just happens to have an interface very very similar to ggplot has in R.

And it looks pretty good from what I've seen of it.

# ? May 22, 2015 16:16

Cingulate: Oct 23, 2012; by Fluffdaddy

outlier posted:

Call that an implied requirement. My main beef with matplotlib (while admitting it's a powerful package lots of people love) is that it just doesn't stick to my brain and feels like I'm swapping to another language when I do plotting. And should you look at the source to see how things work, you're confronted with a whole lot of functions using kwargs.

Doing an R call may be the way to go.

I think that's being a bit too down on matplotlib - you can still do reasonably pythonic stuff that would be rather different in Matlab. Like, iterating over axes to set their properties etc.

# ? May 22, 2015 17:51

vikingstrike: Sep 23, 2007; whats happening, captain

Sitting down over a weekend and reading the docs and tweaking my matplotlibrc file to my liking was one of the best things I ever did. While I went through the docs, I created dummy template files that have the code for the most common plots I do, so now I just copy and paste it over and I'm off to the races. I tried out seaborn and some of the other graphics libraries, but just always came back to matplotlib.

Cingulate posted:

I think that's being a bit too down on matplotlib - you can still do reasonably pythonic stuff that would be rather different in Matlab. Like, iterating over axes to set their properties etc.

Yeah, this is really nice as well. Especially when you are handling multiple subplots in one figure.

# ? May 22, 2015 18:17

QuarkJets: Sep 8, 2008

Cingulate posted:

I think that's being a bit too down on matplotlib - you can still do reasonably pythonic stuff that would be rather different in Matlab. Like, iterating over axes to set their properties etc.

Yes, it's much better than plotting in Matlab, but it could still use some improvement. Better documentation would be a good start, since matplotlib isn't doing much better than Matlab in that regard, and I think that both Matlab and matplotlib are practically impenetrable to new users who want to do something other than the most basic poo poo. Plus all of the aspects of matplotlib that were designed to mimic Matlab resulted in a bunch of methods and functions that are just frustrating to use, and it feels like the more Pythonic side of things was neglected.

For instance, if you create a new Figure, and then want to plot something on an old Figure, how do you do that? There are two solutions as far as I can tell

1) Call figure(N) where N is a magic number referring to the old figure, then call pyplot.plot. This is the Matlab style of doing things and it sucks.

2) Using the old Figure instance, call Figure.add_subplot(1,1,1) to get an Axes object, then call Axes.plot. This is way more Pythonic, but still pretty awkward. Instead, it would be to be able to call Figure.plot, which would return an Axes object. Basically, it shouldn't be necessary to call add_subplot in the case where you only want a single plot on the figure.

I don't take issue with there being a Matlab style of doing things, since the developers wanted to appeal to Matlab users as well as Python users. I take issue with the basic Tutorial offering only the Matlab solution. I also take issue with the Python solution being so clunky, which suggests to me that the Pythonic side of matplotlib has a lot of neglect. It would be really nice if matplotlib provided a "For MATLAB users" tutorial next to a "For PYTHON users" tutorial that would address the respective styles of these very different languages. Instead, most of the tutorials provide the Matlab-style solution, with all of the global imports and magic numbers and horrible bullshit code that you'd expect to see in a Matlab script.

# ? May 22, 2015 18:54

Hughmoris: Apr 21, 2007; Let's go to the abyss!

EDIT:Figured my problem out. In my executemany statement, I had movieList inside the quotes containing the SQL statement, instead of outside. Moving it outside the quotes fixed my problem. I'll leave the problem incase it may help any other beginners.

Ok, I'm stuck. I'm trying to scrape an XML file full of TV show info and save it to a database. Figured it'll teach me a little bit about both. So, I've successfully scraped the XML file and can parse out the Title and the ID. The XML file looks very similar to this, except it holds information on about 50,000 shows.

Since I can successfully scrape and print out the tv show Title and ID, I added code to commit it to an sqlite3 database. When trying to execute the script, I receive the error:

code:

File "python_tvRage_allshows_xml.py", line 15, in <module>
    c.executemany("INSERT INTO tv_shows VALUES (?,?), movieList")
TypeError: function takes exactly 2 arguments (1 given)

I figure it means my list isn't getting split into TITLE and ID when I try and commit it but I can't figure it out. Any tips?

Below is the code I'm trying to execute. Below that is the script I initially used to create my db.

Python code:

import xml.etree.ElementTree as ET
import sqlite3
conn = sqlite3.connect('tvrage.db')
c = conn.cursor()

movieList = []

tree = ET.parse('./Downloads/show_list.xml')
root = tree.getroot()
for show in root.findall('show'):
    title = show.find('name').text
    id = show.find('id').text
    movieList.append([title, id])

c.executemany("INSERT INTO tv_shows VALUES (?,?), movieList")
conn.commit()
conn.close()

Python code:

import sqlite3
conn = sqlite3.connect('tvrage.db')

c = conn.cursor()
c.execute('''CREATE TABLE tv_shows (title TEXT, id TEXT)''')
conn.commit()
conn.close()

Hughmoris fucked around with this message at 03:51 on May 24, 2015

# ? May 24, 2015 03:40

Fergus Mac Roich: Nov 5, 2008; Soiled Meat

Looks like a typo. I believe it'll work if you change:

code:

c.executemany("INSERT INTO tv_shows VALUES (?,?), movieList")

code:

c.executemany("INSERT INTO tv_shows VALUES (?,?)", movieList)

# ? May 24, 2015 03:49

Hughmoris: Apr 21, 2007; Let's go to the abyss!

Fergus Mac Roich posted:

Looks like a typo. I believe it'll work if you change:
code:
c.executemany("INSERT INTO tv_shows VALUES (?,?), movieList")
to
code:
c.executemany("INSERT INTO tv_shows VALUES (?,?)", movieList)

Good eyes! As soon as I submitted it I went back and took a look at the python docs example and caught the mistake. Corrected it and launched the script. It ran just fine. :3:

Thanks.

# ? May 24, 2015 03:52

BigRedDot: Mar 6, 2008

Speaking of Bokeh, we just had a 0.9 release last week. You can now add callback on selections and widgets with a handful of lines of JavaScript so that all kinds of rich interactions are possible without a server (i.e. in static HTML documents). You can see some simple examples in in the Adding Interactions section of the new User's Guide.

FWIW I use to work on Chaco (years ago.. I added contour and image plots to it, among other things). If you need interactive plotting solely for rich clients it's definitely an option, tho I find it a bit excessively fiddly, syntax-wise.

BigRedDot fucked around with this message at 00:18 on May 25, 2015

# ? May 25, 2015 00:15

DimpledChad: May 14, 2002; Rigging elections since '87.

I have a couple years of professional experience as a SW engineer, and I just accepted a new job in a Python shop. I'm familiar with the basics of Python and have written small scripts and ETL stuff with it before, but have never really used it in a professional capacity. I'm looking for one or more books to read to help me get up to speed. I'm not looking for something targeted at beginning programmers, but but rather something that will teach me the ins and outs and how to be "pythonic." I'm not sure which version of Python they use, but I do know that they use Tornado extensively. Any recommendations?

# ? May 26, 2015 19:26

more like dICK: Feb 15, 2010; This is inevitable.

Fluent Python. It's written for Python 3 but calls out the differences from 2, which is what your job will be using.

more like dICK fucked around with this message at 00:20 on May 27, 2015

# ? May 27, 2015 00:17

DimpledChad: May 14, 2002; Rigging elections since '87.

Thanks, I'll check that out. They only started using Python about two years ago, so there is an outside possibility that they may be using 3.

# ? May 27, 2015 03:06

epswing: Nov 4, 2003; Soiled Meat

I seem to be getting "Forbidden 403, CSRF cookie not set" on IE, FIreFox, and Chrome, when running the local dev server. I'm getting the error when submitting the standard django admin site login form. I don't think it's likely that all of a sudden none of the browsers on my machine are accepting cookies. What else could cause this? I can see the csrfmiddlewaretoken hidden input in the html source.

Edit: Django 1.7.7

Edit: vvv Whoops! Thank you.

epswing fucked around with this message at 17:33 on May 28, 2015

# ? May 28, 2015 17:14

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

epalm posted:

I seem to be getting "Forbidden 403, CSRF cookie not set" on IE, FIreFox, and Chrome, when running the local dev server. I'm getting the error when submitting the standard django admin site login form. I don't think it's likely that all of a sudden none of the browsers on my machine are accepting cookies. What else could cause this? I can see the csrfmiddlewaretoken hidden input in the html source.

Edit: Django 1.7.7

FWIW, there's a Django thread.

# ? May 28, 2015 17:29

Hughmoris: Apr 21, 2007; Let's go to the abyss!

*Solved.
I'm stumped on a problem in Flask. I'm trying to test how to submit input and do something with it, then return it to the user. In this case, the user enters a string and submits it, and the webapp returns the string in all uppercase. My problem is understanding the routes and how GET/POST works.

This code right here works as intended:

Python code:

@app.route('/user')
def my_form():
    return render_template("user_input.html")

@app.route('/', methods=['POST'])
def my_form_post():

    text = request.form['text']
    processed_text = text.upper()
    return processed_text

My problem is when I change the approute that handles the POST to be '/user'. I get my initial form to submit my string, but when I submit it I receive the error "The method is not allowed for the requested URL." Here is the code that does NOT work:

Python code:

@app.route('/user')
def my_form():
    return render_template("user_input.html")

@app.route('/user', methods=['POST'])
def my_form_post():

    text = request.form['text']
    processed_text = text.upper()
    return processed_text

I'm hacking this together from an example on stackoverflow so I'm thinking I'm missing a simple concept. Can I get user input and return an output on the same app.route path?

Hughmoris fucked around with this message at 04:05 on May 29, 2015

# ? May 29, 2015 02:17

more like dICK: Feb 15, 2010; This is inevitable.

What does the html for your form look like? Is the "action" attribute pointing to your new route?

more like dICK fucked around with this message at 03:47 on May 29, 2015

# ? May 29, 2015 03:41

Hughmoris: Apr 21, 2007; Let's go to the abyss!

Fergus Mac Roich posted:

Looks like a typo. I believe it'll work if you change:
code:
c.executemany("INSERT INTO tv_shows VALUES (?,?), movieList")
to
code:
c.executemany("INSERT INTO tv_shows VALUES (?,?)", movieList)

Incidentally, this is a good time to mention you should reexamine your choice of editor.

Most editors will use a different color for the contents of python string literals (eg "words inside quotes") so you know when/if you've terminated your quotes properly. In this case you would have quickly seen that the entirety of the code inside c.executemany was a single string, which is probably not what you wanted.

# ? May 31, 2015 21:50

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

ShadowHawk posted:

Incidentally, this is a good time to mention you should reexamine your choice of editor.

Most editors will use a different color for the contents of python string literals (eg "words inside quotes") so you know when/if you've terminated your quotes properly. In this case you would have quickly seen that the entirety of the code inside c.executemany was a single string, which is probably not what you wanted.

Not only that, but PyCharm would specifically switch to an SQL context when the cursor was inside the SQL statement and do syntax highlighting and code completion.

# ? Jun 1, 2015 03:34

Harriet Carker: Jun 2, 2009

Some posters recently recommended I learn about testing, and coincidentally the next Coursera course in the series I'm taking is testing focused. So far so good but one question is bugging me. I know how to run a test suite to test for situations where I know the expected result, but how do I test with randomness? For example, while making a game I initialize a game board and can easily test that the initialization works as expected. Then I add a number to a random square. Using my testing suite, there isn't an expected value for the board to compare to the computed value. Obviously this case is trivial and can easily be tested by just printing the board, but what about for more complex functions? Again, this might be a tough question to tackle in a forums post.

# ? Jun 2, 2015 09:38

Cingulate: Oct 23, 2012; by Fluffdaddy

dantheman650 posted:

Some posters recently recommended I learn about testing, and coincidentally the next Coursera course in the series I'm taking is testing focused. So far so good but one question is bugging me. I know how to run a test suite to test for situations where I know the expected result, but how do I test with randomness? For example, while making a game I initialize a game board and can easily test that the initialization works as expected. Then I add a number to a random square. Using my testing suite, there isn't an expected value for the board to compare to the computed value. Obviously this case is trivial and can easily be tested by just printing the board, but what about for more complex functions? Again, this might be a tough question to tackle in a forums post.

Fix the random seed?

random.seed(0) etc

# ? Jun 2, 2015 09:45

suffix: Jul 27, 2013; Wheeee!

dantheman650 posted:

Some posters recently recommended I learn about testing, and coincidentally the next Coursera course in the series I'm taking is testing focused. So far so good but one question is bugging me. I know how to run a test suite to test for situations where I know the expected result, but how do I test with randomness? For example, while making a game I initialize a game board and can easily test that the initialization works as expected. Then I add a number to a random square. Using my testing suite, there isn't an expected value for the board to compare to the computed value. Obviously this case is trivial and can easily be tested by just printing the board, but what about for more complex functions? Again, this might be a tough question to tackle in a forums post.

Some ways to do it:

Use a fixed random seed and make a test for whatever you happen to get out:
This is pretty fragile, but it can be okay if you have a result that must never change.

Python code:

    random.seed(0)
    vampire.stealLife(player)
    self.assertEqual(old_player_health - player.health, 4)  # We know that randrange returns 4

Make a mock random generator and specify exactly what the test should roll:

Python code:

    with mock.patch.object(random, 'randrange') as mock_randrange:
        mock_randrange.return_value = 4
        vampire.steal_life(player)
        mock_randrange.assert_called_once_with(0, 5)
        self.assertEqual(4, old_player_health - player.health)

When your project gets bigger and you're making abstractions you can hide randomness at the lower levels - so you could make a board initialization function that takes a square picker, and a square picker that takes a random number generator, and when you unit test the board initialization, you make a mock square picker that picks the tile you want.

Or you can go with the randomness and check invariants instead of comparing to an exact result:

Python code:

    vampire.steal_life(player)
    lost_life = old_player_health - player.health
    self.assertGreaterEqual(lost_life, 1, 'Should steal at least one hp')
    self.assertLessEqual(lost_life, 4, 'Should steal at most four hp')
    self.assertEqual(lost_life, vampire_health - old_vampire_health, 'The stolen hp should be added to the vampire')

# ? Jun 2, 2015 21:15

Baby Babbeh: Aug 2, 2005; It's hard to soar with the eagles when you work with Turkeys!!

Does anyone have a good resource for understanding decorators? I'm roughly a month into learning Python and I'm having a hard time wrapping my head around them. They're just an easy way to pass one function to another as an argument? Why do I want to do that?

# ? Jun 2, 2015 23:04

accipter: Sep 12, 2003

Baby Babbeh posted:

Does anyone have a good resource for understanding decorators? I'm roughly a month into learning Python and I'm having a hard time wrapping my head around them. They're just an easy way to pass one function to another as an argument? Why do I want to do that?

Have you read this: http://thecodeship.com/patterns/guide-to-python-function-decorators/ ?

Also, have a look at the Python Decorator Library.

suffix posted:

Thanks for the awesome example of using mock.

# ? Jun 2, 2015 23:20

salisbury shake: Dec 27, 2011

Baby Babbeh posted:

Does anyone have a good resource for understanding decorators? I'm roughly a month into learning Python and I'm having a hard time wrapping my head around them. They're just an easy way to pass one function to another as an argument? Why do I want to do that?

Let's say you're trying to figure out a codebase someone else wrote

Python code:

def dump_args(func: FunctionType) -> FunctionType:
    """
    This might make debugging easier
    """
    @wraps(func)
    def new_func(*args, **kwargs):
        print('CALL', func, 'with args', args, 'and kwargs', kwargs, '\n')
        result = func(*args, **kwargs)
        print('RESULT', func, 'returned', result, 'of type', type(result), '\n')

        return result

    return new_func

Stick a @dump_args on a bunch of methods/functions and you'll get a map of how poo poo flows through the program

Or you have a set of functions that interact w/ a system/session/request/etc with the same setup or tear down behavior

Python code:

def handle_session(func: FunctionType) -> FunctionType:
    """
    Simple decorator function that will handle the session
    for us.
    """

    @wraps(func)
    def new_func(session: scoped_session, *args, **kwargs):
            with session_handler(session) as db_session:
                try:
                    return func(db_session, *args, **kwargs)

                except DB_EXCEPTION as ex:
                    db_session.rollback()

                    raise DatabaseException("Bad stuff") from ex

    return new_func

# now you can do
@handle_session
def insert_row(db_session, row):
    modified_row = do_stuff(row)
    db_session.merge(modified_row)

@handle_session
def delete_row(db_session, row):
    db_session.delete(row)

# instead of 
def insert_row(session, row):
    with session_handler(session) as db_session:
           try:
                modified_row = do_stuff(row)
                db_session.merge(modified_row)

            except DB_EXCEPTION as ex:
                db_session.rollback()

                raise DatabaseException("Bad stuff") from ex

# etc

They let you factor out common behavior that a set of functions share. You can stick a @handle_session on a function and not have to deal with a context manager or handling certain exceptions over and over again in each function body.

---

PEP-0492: async & await and PEP-0484: type hints have been accepted.

Good: Python is becoming slow C#
Bad: Comments can mean something other than "ignore everything after the # until a new line"

salisbury shake fucked around with this message at 07:45 on Jun 4, 2015

# ? Jun 3, 2015 19:01

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

salisbury shake posted:

Bad: Comments can mean something other than "ignore everything after the # until a new line"

I think everyone involved acknowledges that this isn't the best possible solution, but getting PEP-0484 accepted for 3.5 wasn't going to happen with new syntax and supporting 3.4 was definitely not going to happen with new syntax.

Guido seems to favor new syntax for this in 3.6.

# ? Jun 3, 2015 19:44

Baby Babbeh: Aug 2, 2005; It's hard to soar with the eagles when you work with Turkeys!!

Thanks for the help, accipter & salisbury shake. That makes it a bit clearer to me. I think I wasn't getting it because the test code I was given to explain it was just a useless function that demonstrated that the original function wouldn't run, so I couldn't see why I'd want to write a decorator like that rather than just modifying the original function. Decorators seem like they could be really handy!

# ? Jun 3, 2015 20:29

Cingulate: Oct 23, 2012; by Fluffdaddy

salisbury shake posted:

Stick a @dump_args on a bunch of methods/functions and you'll get a map of how poo poo flows through the program

Okay now I get it.

# ? Jun 3, 2015 21:29

SurgicalOntologist: Jun 17, 2004

That said, it is pretty much just convenient syntax. You could equivalently add

Python code:

some_func = dump_args(some_func)

for each function. The decorator syntax turns this from a weird trick into a readable language feature, even if under the hood it's the same thing.

Someone who has read PEP492 and understands asyncio better than I do: what's the significance for this with asnycio applications? Is it basically replacing yield from with await to make the meaning more transparent? Or does it include deeper changes?

# ? Jun 4, 2015 16:47

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

SurgicalOntologist posted:

Is it basically replacing yield from with await to make the meaning more transparent?

Yes. Using @asyncio.coroutine and yield from will continue to work as before, though.

It's kind of a weird thing they did, but I guess they wanted to not introduce any new keywords for 3.4.

# ? Jun 4, 2015 17:35

Hughmoris: Apr 21, 2007; Let's go to the abyss!

Is there any downside for the average Python hobbyist to use Anaconda? It looks like it comes with a ton of great modules.

# ? Jun 7, 2015 00:13

Nippashish: Nov 2, 2005; Let me see you dance!

Hughmoris posted:

Is there any downside for the average Python hobbyist to use Anaconda? It looks like it comes with a ton of great modules.

No downside, but if you use python properly (ie. a virtualenv for each project) there's not much upside either.

# ? Jun 7, 2015 00:17

QuarkJets: Sep 8, 2008

Hughmoris posted:

Is there any downside for the average Python hobbyist to use Anaconda? It looks like it comes with a ton of great modules.

No.

I guess the default install is kind of huge, like a couple of GB. So if you're using a 20 year-old machine, that could be an issue.

# ? Jun 7, 2015 00:18

Dominoes: Sep 20, 2007

Hughmoris posted:

Is there any downside for the average Python hobbyist to use Anaconda? It looks like it comes with a ton of great modules.

It doesn't play well with Qt5.

# ? Jun 7, 2015 00:19

accipter: Sep 12, 2003

Nippashish posted:

No downside, but if you use python properly (ie. a virtualenv for each project) there's not much upside either.

I would say the upside of Conda/Anaconda is ease of installation in Windows.

# ? Jun 7, 2015 16:41

Nippashish: Nov 2, 2005; Let me see you dance!

accipter posted:

I would say the upside of Conda/Anaconda is ease of installation in Windows.

I didn't think of that because I never use python on windows, but that is a good point.

# ? Jun 7, 2015 18:48

Adbot: ADBOT LOVES YOU

# ? Jun 3, 2024 16:18

pmchem: Jan 22, 2010

accipter posted:

I would say the upside of Conda/Anaconda is ease of installation in Windows.

Anaconda is also great for enterprise environments where getting Anaconda approved is one piece of paperwork, and you can't run "pip install" etc., because instead you're supposed to do more paperwork for each module.

# ? Jun 8, 2015 01:44

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

code:

Python code:

Python code:

code:

code:

code:

code:

Python code:

Python code:

HTML code:

code:

code:

Python code:

Python code:

Python code:

Python code:

Python code:

Python code: