Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
FoiledAgain
May 6, 2007

I don't have any real formal background in math either, so it's entirely possible that your idea will work. It seemed like the wrong approach to me, because you aren't interested in finding the shortest path. Even if you select a random adjacent number, it will still take you the exact same number of steps to get down the pyramid. But now that I re-read your answer, I think I see your idea: you want the number in the pyramid to represent the distance. Bigger numbers are "closer", so you look for the "shortest" path in that sense. That seems workable (but then see my first sentence).

Adbot
ADBOT LOVES YOU

KICK BAMA KICK
Mar 2, 2009

Grain of salt cause I only know this from a Coursera algorithms class that just covered the topic this week, but yeah, I think he's looking for the shortest path in a weighted directed acyclic graph. Djikstra's algorithm isn't wrong, but if since we can stipulate that it's acyclic there's an easier solution based on topological sorting.

e: again, I just barely understand this stuff, but I'm guessing the key might be that you can omit the actual work of doing the topological sort since the graph is structured the way it is?

KICK BAMA KICK fucked around with this message at 04:47 on Nov 11, 2015

baka kaba
Jul 19, 2003

PLEASE ASK ME, THE SELF-PROFESSED NO #1 PAUL CATTERMOLE FAN IN THE SOMETHING AWFUL S-CLUB 7 MEGATHREAD, TO NAME A SINGLE SONG BY HIS EXCELLENT NU-METAL SIDE PROJECT, SKUA, AND IF I CAN'T PLEASE TELL ME TO
EAT SHIT

Fergus Mac Roich posted:

Yeah... it's still not intuitive to me why this algorithm can't do this(note that my script does model the constraint you mentioned) but I guess I should have read further. There are other internet resources making it clear that this algorithm is just inapplicable to longest path in a DAG, assuming that I'm right that this problem actually can be modeled in that way. I guess I'll come back to this problem one day when I understand graph algorithms better and maybe it'll make more sense. At least I have a solution. Thanks.

I think your issue is something to do with this:



From looking at this, it's obvious the longest path has to go through both 20s and that 8




Because of the way the numbers fall, that heavy 20 on the left hasn't had its distance set yet, because the 1 above it hasn't been visited. But we've already set both the distances of the other 20's neighbours, including the 8 at the bottom. Because they have the heaviest distances, they're going to get visited next and get removed from the unvisited group.




There's barely anything left unvisited, and only now are we hitting that 20




The 20 checks its neighbours and adds its distance to them - the other 20 gets bumped up to 43 (in your implementation - Djikstra wouldn't do this, it only checks unvisited nodes, but it doesn't matter). What it really needs to do is propagate that to its neighbours, so the 8 can be updated to 51. But because the lower 20 has already been visited, it won't change its neighbours again. The last unvisited nodes are calculated, and the highest distance is 43 - which is wrong, and isn't from a node at the bottom of the graph either

I think your issue is basically that your graph is one-way - you can only move downwards, which limits the ability of the algorithm to reach out to its adjoining nodes and keep a consistent smallest/biggest distance score. You only get one chance to visit a node, and once it's visited its distance can't be updated (in Djikstra's algorithm anyway - yours updates neighbours whether they're visited or not as far as I can see), but this pyramid has separate, distinct routes into each node since there's no backtracking, so each one needs to be evaluated. With Djikstra it's down to luck whether when you visit a node it has all its incoming route information in place, because you can't go upwards looking for it, if you get me


If you're interested, this is how it plays out (I think...) without the 'downwards only' neighbour check, but still looking for the largest route instead of the smallest (red is going up, against the rules)



So yeah, it looks like going for the heavier weight works fine, it's just the limitation on the direction you can go that makes it a bad fit


(sorry for the megapost, I couldn't see anything wrong at first so I got curious)

Emacs Headroom
Aug 2, 2003
I didn't check your implementation, but I think Dijkstra's would work if you set up the graph structure correctly (i.e. in the triangle every node has two children in the level below it, except for the final layer which all feeds into a "sink" node that is your target).

The solution is going to be dynamic programming of some sort; my first pass would be somethng like:

1. Assign each node in the tree DAG a weight w_n equal to the intial weight supplied

2. Starting at the lowest level and moving up, for each node n in the level, set w_n = w_n + max(w_m for m in children)

The number at the top of the node should be the number you want after this process is finished (this might be the same as the topological sorting mentioned earlier, but I'm too lazy / hung over to check)

Fergus Mac Roich
Nov 5, 2008

Soiled Meat

Emacs Headroom posted:

I didn't check your implementation, but I think Dijkstra's would work if you set up the graph structure correctly (i.e. in the triangle every node has two children in the level below it, except for the final layer which all feeds into a "sink" node that is your target).

The solution is going to be dynamic programming of some sort; my first pass would be somethng like:

1. Assign each node in the tree DAG a weight w_n equal to the intial weight supplied

2. Starting at the lowest level and moving up, for each node n in the level, set w_n = w_n + max(w_m for m in children)

The number at the top of the node should be the number you want after this process is finished (this might be the same as the topological sorting mentioned earlier, but I'm too lazy / hung over to check)

I'm not sure if that's topological sorting because I haven't studied it yet(thanks for the heads up to whoever posted it, I am definitely going to read up), but that's actually pretty much the solution I ended up with.


baka kaba posted:

(sorry for the megapost, I couldn't see anything wrong at first so I got curious)

This post is great and I feel like it improved my understanding of the algorithm, so thank you.

KernelSlanders
May 27, 2013

Rogue operating systems on occasion spread lies and rumors about me.
If not for the downward only constraint the longest path is any of the many possibilities that touch every node. For the downward only version, why wouldn't you just start at the bottom and work back to the top? For the row one up from the bottom, each node is worth its weight plus the greater of the two below it. Assign that as the new weight for the node and repeat for the row above. O(n)

Cingulate
Oct 23, 2012

by Fluffdaddy

Cingulate posted:

Ah well, my 3.5/Anaconda/MKL woes continue. ...
Okay this was just stupid. Anaconda simply hadn't yet made an mkl version of numpy available. As of today, it's in the standard repos though.

PoizenJam
Dec 2, 2006

Damn!!!
It's PoizenJam!!!
So Psychopy's documentation is utterly loving miserable and I can't find the answer to this. So a part of my code for a Psych experiment goes like this:

code:
    core.wait(2, hogCPUperiod=2)
    acc_resp_raw = event.getKeys('num_1,num_2,num_3') 
    if acc_resp_raw[0] == 'num_1':
        acc_resp = '1'
    elif acc_resp_raw[0] == 'num_2':
        acc_resp = '2'
    elif acc_resp_raw[0] == 'num_3':
        acc_resp = '3'
Basically, following a trial presentation, the stimuli disappears and a blank screen appears for two seconds. During the 2 seconds, I have a window to 'code' the experiment, in case they hosed up or got it incorrect, so I can eliminate that trial from analysis.

So it generally works fine if I make a keypress, however if no keypress is made in the time-window the program crashes with an 'IndexError: list index out of range' error.

Instead, what I'd like for it to do is 'acc_resp = '1'' if no keypress is detected. But I cannot find the appropriate documentation regarding this.


Edit: I feel silly. I solved this one on my own moments after posting this. When I was testing solutions, I kept putting the check for the empty list last in the stack of conditions. It needs to come first.

For anyone interested, the solution was:

code:
    core.wait(2, hogCPUperiod=2)
    acc_resp_raw = event.getKeys('num_1,num_2,num_3') 
    if not acc_resp_raw:
        acc_resp = '1'
    elif acc_resp_raw[0] == 'num_1':
        acc_resp = '1'
    elif acc_resp_raw[0] == 'num_2':
        acc_resp = '2'
    elif acc_resp_raw[0] == 'num_3':
        acc_resp = '3' #change for benq

PoizenJam fucked around with this message at 19:00 on Nov 12, 2015

Cingulate
Oct 23, 2012

by Fluffdaddy
How about ...
code:
switch_dict = {"num_{}".format(ii):str(ii) for ii in (1, 2, 3)}
...
core.wait(2, hogCPUperiod=2)
acc_resp_raw = event.getKeys('num_1,num_2,num_3') 
acc_resp = ('1' if not acc_resp_raw else switch_dict[acc_resp_raw[0]])
Although to be honest I have no idea what argument you're feeding to getKeys there.

Also yes Psychopy is the worst-documented, and generally worst, project I'm using. Still, so much better than the alternatives.

PoizenJam
Dec 2, 2006

Damn!!!
It's PoizenJam!!!
That solution is beautifully elegant. Python is all quite new to me, only other programming experience I've had is Visual Basic.

Bob Morales
Aug 18, 2006


Just wear the fucking mask, Bob

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

What's the preferred way to interface with DB2 databases? PyDB2 looks pretty out of date.

Jo
Jan 24, 2005

:allears:
Soiled Meat
Good lord I am absolutely in love with TensorFlow. :3: I'm hoping this isn't just the honeymoon phase before everything falls apart, but everything about it feels better engineered and more Python than Theano with no loss of generality. Has anyone else played with it?

Lysidas
Jul 26, 2002

John Diefenbaker is a madman who thinks he's John Diefenbaker.
Pillbug
I'll be happy to play with it once this is fixed: https://github.com/tensorflow/tensorflow/issues/1

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Lysidas posted:

I'll be happy to play with it once this is fixed: https://github.com/tensorflow/tensorflow/issues/1

It's funny how many issues end up having to have someone telling everyone to stop posting +1 or whatever. I mean, come on you people are supposed to be developers!

EAT THE EGGS RICOLA
May 29, 2008

Bob Morales posted:

What's the preferred way to interface with DB2 databases? PyDB2 looks pretty out of date.

SQLAlchemy and https://github.com/ibmdb/python-ibmdb

obstipator
Nov 8, 2009

by FactsAreUseless
Is there a way to search all pip libraries ordered by most popular? Right now I'm just trying to find which unit testing library is the best for flask, but with other languages I like to browse the most downloaded to see what groupthink has deemed the most useful packages.

Communist Pie
Mar 11, 2007

...Robot!

obstipator posted:

Is there a way to search all pip libraries ordered by most popular? Right now I'm just trying to find which unit testing library is the best for flask, but with other languages I like to browse the most downloaded to see what groupthink has deemed the most useful packages.

http://pypi-ranking.info/search/test/

obstipator
Nov 8, 2009

by FactsAreUseless

Sweeet, thanks!

Cingulate
Oct 23, 2012

by Fluffdaddy
I have a terribly inefficient Pandas loop. Is there any way to optimize this?

I have a range of long-ish data frames. They look like this:

code:
df.head(10)
   Unnamed: 0  POSITION    WORDFORM       LEMMA  CPOS   POS       MORPHOLOGY  \
0           0         1         Der         die   ART   ART  Def|Masc|Nom|Sg
1           1         2  Todesgruss  Todesgruss     N    NN      Masc|Nom|Sg
2           2         3         der         die   ART   ART     Def|_|Gen|Pl
3           3         4    Legionen      Legion     N    NN         _|Gen|Pl
4           4         5      Gregor      Gregor     N    NE         _|Gen|Pl
5           5         6     Samarow     Samarow     N    NE         _|Gen|Pl
6           6         7           .           .    $.    $.                _
7           7         1     Dritter       dritt  ADJA  ADJA     _|_|_|Sg|St|
8           8         2        Band        Band     N    NN           _|_|Sg
9           9         3           .           .    $.    $.                _
They contain sentence parses, and I want to give each sentence a unique sentence ID. The sentences can be distinguished by discontinuities in the POSITION field; for each sentence, words are counted continuously, so if the counter in POSITION resets, I know a new sentence has started. (No, I can't use periods, as they are not reliably found in the input files.)

I'm using this for loop to assign IDs:
code:
for ii, file in enumerate(allfiles):
    print(ii, file)
    df = pd.read_csv(file)
    if "sentence_length" not in df.columns:  # exclude the ones I've already done
        sn, sn_old = 0, 0
        for ii, line in df.iterrows():
            if sn_old > line["POSITION"]:
                sn += 1
            df.loc[ii, "sentence_id"] = sn
            sn_old = line["POSITION"]
This takes ages. I am sure there is a smarter way. Right?

SurgicalOntologist
Jun 17, 2004

There may be a more direct way, but this is the first alternative that comes to mind:

Python code:
df['sentence_id'] = None
sentence_starts = df['POSITION'] == 1
df.loc[sentence_starts, 'sentence_id'] = np.arange(sentence_starts.sum())
df['sentence_id'] = df['sentence_id'].fillna(method='ffill').astype(int) 
The last part is necessary because the first line (the None assignment) will create a series of floats instead of ints.

SurgicalOntologist fucked around with this message at 23:29 on Nov 20, 2015

Dominoes
Sep 20, 2007

Consider using numpy for calculations after reading the csv; pandas is orders of magnitude slower.

Cingulate
Oct 23, 2012

by Fluffdaddy

SurgicalOntologist posted:

There may be a more direct way, but this is the first alternative that comes to mind:

Python code:
df['sentence_id'] = None
sentence_starts = df['POSITION'] == 1
df.loc[sentence_starts, 'sentence_id'] = np.arange(sentence_starts.sum())
df['sentence_id'] = df['sentence_id'].fillna(method='ffill').astype(int) 
The last part is necessary because the first line (the None assignment) will create a series of floats instead of ints.
Yes, that should do it. Thanks!

Dominoes posted:

Consider using numpy for calculations after reading the csv; pandas is orders of magnitude slower.
What? There aren't any actual calculations.

Nippashish
Nov 2, 2005

Let me see you dance!

Dominoes posted:

Consider using numpy for calculations after reading the csv; pandas is orders of magnitude slower.

Numeric columns in pandas are stored as numpy arrays.

SurgicalOntologist
Jun 17, 2004

The indexing is slower though. There is a tradeoff for the convenience. In Cingulate's case, we sped up the operation by indexing only once with a boolean array instead of performing a lookup for every row. Perhaps further speed up could be made by doing all the indexing with numpy instead of pandas but I doubt it is necessary here.

Cingulate
Oct 23, 2012

by Fluffdaddy
I thought that was some fairly readable and elegant code myself.

SAVE-LISP-AND-DIE
Nov 4, 2010
I'm researching how to (ab?)use 3.5's asyncio coroutines for complex event processing but I'm having trouble putting it all together.

Pre "async" & "await", I had a half cocked idea of using the coroutine pattern from generators in place of a state machine. The primary mechanism was having multiple value = (yield)/generator .send() entry-points to my generator function. E.g.
code:
def detect_bad_happening(output=None):
	while True:
		event = (yield)

		if event ==  "bad_can_happen_threshold":
			while True:
				event = (yield)
				if event == "bad_can't_happen_threshold":
					break
				elif event == "foo":
					output.send("BAD HAPPENING")
		else:
			output.send("not bad happening")

def printer():
	while True:
		value = (yield)
		print(value)

detector = detect_bad_happening(output=printer())
next(detector)

detector.send("bad_can_happen_threshold")
# "BAD HAPPENING"
detector.send("foo")

detector.send("bad_can_happen_threshold")
detector.send("bad_can't_happen_threshold")
# "not bad happening"
detector.send("foo")
Is there an equivalent, or better pattern, to this sort of generator usage with asyncio?

duck monster
Dec 15, 2004

Reposting this here because it might be of interest to those who also believe Python is the language of the master race, and javascript on mobile devices needs to die:

code:
********* Start testing of TestPyOtherSide *********
Config: Using QtTest library 5.5.1, Qt 5.5.1 (i386-little_endian-ilp32 static debug build; by Clang 6.0 (clang-600.0.56) (Apple))
PASS   : TestPyOtherSide::initTestCase()
PASS   : TestPyOtherSide::testEvaluate()
PASS   : TestPyOtherSide::testQVariantConverter()
PASS   : TestPyOtherSide::testPyObjectConverter()
PASS   : TestPyOtherSide::testPyObjectRefRoundTrip()
PASS   : TestPyOtherSide::testPyObjectRefAssignment()
PASS   : TestPyOtherSide::testQObjectRef()
PASS   : TestPyOtherSide::testConvertToPythonAndBack()
PASS   : TestPyOtherSide::testSetToList()
PASS   : TestPyOtherSide::cleanupTestCase()
Totals: 10 passed, 0 failed, 0 skipped, 0 blacklisted
********* Finished testing of TestPyOtherSide *********
I'm in the process of testing something very very awesome. Theres about to be a new sheriff in town for dynamic languages and cross platform mobile dev...

I'll let you all know more when I finish, but my gently caress javascript quest is heading towards glory right now.

edit: BAM
code:
********* Start testing of TestPyOtherSide *********
Config: Using QtTest library 5.5.1, Qt 5.5.1 (arm64-little_endian-lp64 static debug build; by Clang 6.0 (clang-600.0.56) (Apple))
PASS   : TestPyOtherSide::initTestCase()
PASS   : TestPyOtherSide::testEvaluate()
PASS   : TestPyOtherSide::testQVariantConverter()
PASS   : TestPyOtherSide::testPyObjectConverter()
PASS   : TestPyOtherSide::testPyObjectRefRoundTrip()
PASS   : TestPyOtherSide::testPyObjectRefAssignment()
PASS   : TestPyOtherSide::testQObjectRef()
PASS   : TestPyOtherSide::testConvertToPythonAndBack()
PASS   : TestPyOtherSide::testSetToList()
PASS   : TestPyOtherSide::cleanupTestCase()
Totals: 10 passed, 0 failed, 0 skipped, 0 blacklisted
********* Finished testing of TestPyOtherSide *********
On the simulator it was using the macs system PYTHONPATH. So I threw in a zip of the standard python library into the test suite and passed a PYTHONPATH environment variable in, and it worked. Well, sort of worked.

Remaining tasks:
1) Scour the zip etc for dylibs or .so files to make sure I'm not being apple violating.,
2) If there are any, modify my build chain to statically link them, or if they look apple violating , remove them all together.
3) Integrate this all into QT Designers tool chain.
4) Party. Hard.

None of this is made easier by the fact that QT's brand of C++ is completely perplexing, but I'm getting there.

duck monster fucked around with this message at 13:43 on Nov 25, 2015

Space Kablooey
May 6, 2009


duck monster posted:

Reposting this here because it might be of interest to those who also believe Python is the language of the master race, and javascript on mobile devices needs to die:


I'm intrigued, please tell me more.

duck monster
Dec 15, 2004

Its basically importing pyotherside which is a QT binding to python that works with QT5 , and hence works on mobile devices, and isnt PyQt (GPL or $$$$$). The basic difference is, PyQt hosts QT in a python program, pyotherside hosts python in a QML program. There where Android ports and a port to "SailfishOS", whatever that is. But no IOS

So I've been hacking away trying to get it to work on the iphone. I'm basically at the "Hello world" stage of it passing unit tests. Now its just a case of getting it to work with QT Designer projects.

Because gently caress javascript. This is a project fueled by hate. (I've spent the last month staring into the abyss of Angular.JS gently caress.that.poo poo its time for python on the telephone)

duck monster fucked around with this message at 14:58 on Nov 25, 2015

Munkeymon
Aug 14, 2003

Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.



Could this also be made to run on Android with NDK? You could have a neat cross-platform kit if you got that figured out, but just getting it working on iOS would be plenty cool all on its own.

duck monster
Dec 15, 2004

Munkeymon posted:

Could this also be made to run on Android with NDK? You could have a neat cross-platform kit if you got that figured out, but just getting it working on iOS would be plenty cool all on its own.

I believe it already runs on Android.

Munkeymon
Aug 14, 2003

Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.



duck monster posted:

I believe it already runs on Android.

Ah, so it does. Not sure why that didn't come up when I googled earlier.

E: google auto-corrects pyotherside to otherside haha thanks goog

Munkeymon fucked around with this message at 18:53 on Nov 25, 2015

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

If you were looking at Angular, its no surprise it drove you to run away from JS. Angular is not very good.

Well, I take that back, its better than most of what came before it (putting aside Angular's atrocious documentation...both official and unofficial. It's amazing to me how bad the situation is with Angular and documentation and best practices.), but there's (IMO) much better alternatives available nowadays like React+redux.

I think Suspicious Dish first told me this, and I now agree with him: javascript is very similar to python.

Munkeymon
Aug 14, 2003

Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.



Thermopyle posted:

I think Suspicious Dish first told me this, and I now agree with him: javascript is very similar to python.

And becoming more and more so with all the new features they're adding.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
Yeah. We were using Mozilla's "JavaScript 2.0" back in 2008 for writing desktop applications, and even back then, the simple additions of let and destructuring assignment made me feel a lot more at home as a former Python guy.

Traceur sort of happened but Google never pushed it, and then Node.js, then Browserify, then Babel, then Webpack happened. I've really been enjoying the Babel / Webpack experience, and I've looked into porting Xplain to it as a trial run.

There's still plenty of whacko in JavaScript, like eval, but it's isolated enough not to matter.

German Joey
Dec 18, 2004

duck monster posted:

Because gently caress javascript. This is a project fueled by hate.

bless you, bless you.

hooah
Feb 6, 2006
WTF?
My project partner's turning out to be either a lazy sack of poo poo or in way over his head in terms of time management, so I'm having to work around that. I have a Python program that calls a jar file which spits some stuff back out. It would be nice for it to just spit out one number representing what we're interested in, but that's what my partner hasn't done for two or three weeks. So, I figure I can get around it by counting the lines in the output, since that'll roughly correlate to what we want. However, I want to stop counting newline characters when Python sees a "=" in the output. Is there a neat way to do that?

Cingulate
Oct 23, 2012

by Fluffdaddy

hooah posted:

My project partner's turning out to be either a lazy sack of poo poo or in way over his head in terms of time management, so I'm having to work around that. I have a Python program that calls a jar file which spits some stuff back out. It would be nice for it to just spit out one number representing what we're interested in, but that's what my partner hasn't done for two or three weeks. So, I figure I can get around it by counting the lines in the output, since that'll roughly correlate to what we want. However, I want to stop counting newline characters when Python sees a "=" in the output. Is there a neat way to do that?
You mean something like

input.split("=")[0].count("\n")

hooah
Feb 6, 2006
WTF?
Ooh, yes, exactly like that, thanks!

Adbot
ADBOT LOVES YOU

Cingulate
Oct 23, 2012

by Fluffdaddy
Or this should also work:

input[:input.index("=")].count("\n")

(.index finds the first occurrence)

I guess if these files are very long or you call that very often, you'd choose a method that doesn't take the entire file. But this should work for now.

  • Locked thread