The Usenet Megathread - No invite discussion!

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Usenet Megathread - No invite discussion!

«‹›357 »

Diamonds On MY Fish: Dec 10, 2008; I WAS BORN THIS WAY

Shy posted:

What's a good software to browse text? Specifically big groups, Thunderbird seemed too clumsy.

They have TEXT groups on Usenet now?

# ? Dec 20, 2012 19:56

Adbot: ADBOT LOVES YOU

# ? Apr 25, 2024 19:59

Shy: Mar 20, 2010

drat you guys.

# ? Dec 20, 2012 20:34

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Shy posted:

drat you guys.

In case you (or I) weren't (wasn't) clear: Google Groups is a real answer.

# ? Dec 20, 2012 21:36

Shy: Mar 20, 2010

Thermopyle posted:

In case you (or I) weren't (wasn't) clear: Google Groups is a real answer.

Yeah I know, thanks. I was thinking about a classic application I could run from my connection, Google might cover it better though.

# ? Dec 20, 2012 21:43

Squibbles: Aug 24, 2000; Mwaha ha HA ha!

Shy posted:

Yeah I know, thanks. I was thinking about a classic application I could run from my connection, Google might cover it better though.

Outlook express used to do it. Not sure if it still does though.

# ? Dec 20, 2012 21:44

Potassium Problems: Sep 28, 2001

https://github.com/strider-/Nizbel/

I'm making a Rails engine so you could hook this up into an existing app. I banged it out in my free time over a week so it's pretty crude right now, but it's got support for SSL/header compression. The NNTP classes are here: https://github.com/strider-/Nizbel/tree/master/lib/nizbel/nntp

# ? Dec 20, 2012 21:54

Sperg Victorious: Mar 25, 2011

Shy posted:

What's a good software to browse text? Specifically big groups, Thunderbird seemed too clumsy.

40Tude Dialog. It's kinda old, and you'll need to install it somewhere thats not in "program files" for it to work, but it's one of the better ones.

# ? Dec 20, 2012 22:00

GhostSeven: Apr 23, 2005; Yesterday Was A Million Years Ago

Thermopyle posted:

Oh, nice. Thanks. I was dreading looking through the code to fix this myself. You might want to send them a pull request on github.

Yea it was not too bad to find actually, just slapped some logging on various places and it was ok to trace back, code is fairly coherent and mainly commented in English which is nice.

I am not sure they will want that pull request as their current search style appears to be by design. Severely broken for api calls that try to find old files though, as it just presents a mass of data that is not relevant.

You know I have never done a pull request, how would one go about it?

# ? Dec 20, 2012 22:04

Shy: Mar 20, 2010

Squibbles posted:

Outlook express used to do it. Not sure if it still does though.

That newer Mail app could do it too, but it was really slow last time I tried.

Sperg Victorious posted:

40Tude Dialog. It's kinda old, and you'll need to install it somewhere thats not in "program files" for it to work, but it's one of the better ones.

Nice, never heard of it. Thank you!

# ? Dec 20, 2012 22:05

GhostSeven: Apr 23, 2005; Yesterday Was A Million Years Ago

Thermopyle posted:

You might want to send them a pull request on github.

Figured it out, git hub is quite fun, pull request submitted, maybe they will re-think their approach and apply the "fix"

# ? Dec 20, 2012 22:48

Diamonds On MY Fish: Dec 10, 2008; I WAS BORN THIS WAY

I used to use Free Agent by Forte back in the day for text groups, but it looks like they don't make it anymore

# ? Dec 20, 2012 22:57

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

GhostSeven posted:

Figured it out, git hub is quite fun, pull request submitted, maybe they will re-think their approach and apply the "fix"

Nicely done. Yay for open source.

BTW, I'm lazy and looking at PHP makes me die a little bit...what's "hacky" about your solution?

# ? Dec 21, 2012 00:04

Dude With Pants: May 29, 2010

GhostSeven posted:

?? nzbs.org has games both console and pc / mac, TV and movies...

Yes but remember how nzbmatrix listed everything in categories within the browse page ? Most newznab index sites don't do that for you , so you have to search for your self within each category to see what is released .

some other useful index sites

nzbsa.co.za

nzb7.com

# ? Dec 21, 2012 01:06

Ronald Duck: Jun 26, 2005; My mum says I’m cool.

Dude With Pants posted:

nzbs.org is also good but they don't have top 10 movies , or games , or tv .

Browse a group, hit the "Top NZBs []24h []1 week []1 month" option.

kiresays posted:

What's the difference between spotweb/newznab and an indexer site?

Spotweb/newznab is the back-end software that runs an index site. Nzbs.org/Nzbs.su use newznab.

Newznab creates NZB files automatically by reading all the posts added to any group that it is monitoring. With the huge amount of posts added to newsgroups daily it takes a considerable amount of time and CPU power to churn through that data.

Spotweb has NZB files created by humans which are posted to a newsgroup along with any user comments as well. That means that any Spotweb site has the exact same NZB files and user comments.

For my server running Newznab to get 7 days worth of posts in a handful of groups it took several hours. In < 24 hours running Spotweb I have 2 years worth of NZB files along with heaps of user comments as well, however most of the content and user comments are Dutch.

If you have the CPU power and the disk space (and don't speak Dutch) right now Newznab is much more useful. However, all Spotweb really needs is 1 decent sized English speaking community to use it and it could be awesome for everyone. I still think any hypothetical Goon indexer should use Newznab, it creates a much better site and there's more content.

Ronald Duck fucked around with this message at 01:18 on Dec 21, 2012

# ? Dec 21, 2012 01:14

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Chick3n posted:

I still think any hypothetical Goon indexer should use Newznab, it creates a much better site and there's more content.

I think a hypothetical goon indexer should be something custom. Maybe using the spotnet protocol that also does automatic usenet parsing but stores its data on spotnet or a custom protocol based on usenet.

It's not a terribly hard project.

In fact, I'll say right now, that I'll collaborate on developing such an indxer. Don't have the time to do it all, but if it's in Python (the only appropriate language that I enjoy coding with for fun projects) I'll help in planning and contributing code.

# ? Dec 21, 2012 01:51

A Wealthy Refugee: Jan 4, 2003; by Y Kant Ozma Post

And double xp !

# ? Dec 21, 2012 01:54

Telex: Feb 11, 2003

Meanwhile, if anyone's managed to get newznab+ running on a Nas4Free/Freenas host, please explain how you got the white pages on search to go away.

I'm at the end of my expertise here.

# ? Dec 21, 2012 01:55

movax: Aug 30, 2008

Thermopyle posted:

I think a hypothetical goon indexer should be something custom. Maybe using the spotnet protocol that also does automatic usenet parsing but stores its data on spotnet or a custom protocol based on usenet.

It's not a terribly hard project.

In fact, I'll say right now, that I'll collaborate on developing such an indxer. Don't have the time to do it all, but if it's in Python (the only appropriate language that I enjoy coding with for fun projects) I'll help in planning and contributing code.

Yeah you should poke me on IRC.

# ? Dec 21, 2012 02:05

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

movax posted:

Yeah you should poke me on IRC.

I just poked the hell outta you and nothing happened. I'm quite disappointed.

# ? Dec 21, 2012 02:43

gary oldmans diary: Sep 26, 2005

Spotweb sounds awfully familiar. From the wake of the court ruling against Newzbin (1) before it went down:

xlevus posted:

Why not have editors post reports to usenet themselves in a community-accepted, plaintext, machine-readable format? Sort of like an extended NZB. Make the code to index it freely available and any joe can fire his clone and share the data. In effect de-centralizing Newzbin.

Admittedly, you'd lose the ability to edit reports, user comments and semi-verified integrity, but these issues could be worked around to some degree.

poo poo, Tempted to code this myself.

_{xlevus hosed around with this message at Mar 30, 2010 around 06:15}

I guess we missed the boat on this idea as it was being floated as a peer-contributed source to go alongside automated scrapers. I recommended public key signing to allow identification of favored and disfavored nzb contributors and said we should implement this before the Dutch do.
The Dutch! :argh:

# ? Dec 21, 2012 05:04

Evil Trout: Nov 16, 2004; The evilest trout of them all

Lone_Strider posted:

https://github.com/strider-/Nizbel/

I'm making a Rails engine so you could hook this up into an existing app. I banged it out in my free time over a week so it's pretty crude right now, but it's got support for SSL/header compression. The NNTP classes are here: https://github.com/strider-/Nizbel/tree/master/lib/nizbel/nntp

Looks awesome. I'll look in more detail later.

# ? Dec 21, 2012 05:20

Ronald Duck: Jun 26, 2005; My mum says I’m cool.

Hogburto posted:

I guess we missed the boat on this idea as it was being floated as a peer-contributed source to go alongside automated scrapers.

If only Nzbmatrix and Newzbin2 had used Spotweb

. Someone needs to convince the Nzbmatrix refugees to setup a Spotweb site and start posting.

# ? Dec 21, 2012 10:22

Lusername: Sep 22, 2005; The truth is just an excuse for a lack of imagination.

Spotweb sounds very promising if it's adopted by the English-speaking community. But since it's decentralized, how susceptible is it to being 'poisoned' by outside organisations? By poisoning I mean things like spamming every release with dozens of fake duplicates, or perhaps trying to destroy the trust in the user generated comments by automatically posting bogus ones like 'unrepairable', 'missing rars', 'already taken down on Astraweb+BlockNews', 'hardcoded russian subs', 'goes out of sync at 23m:11s', 'cropped aspect ratio', etc.?

# ? Dec 21, 2012 11:46

longview: Dec 25, 2006; heh.

GhostSeven posted:

I have just fixed that as it was winding me up!

Assuming you are using mysql this is the hacky fix to essentially default to an AND search if you have not specified any other search modifiers such as + - etc.

It also works for newznab api so improves Sickbeard et al.

It is hacky so be warned!

Edit /spotweb-install-dir/lib/dbeng/dbfts_mysql.php

BACK UP THE FILE!

Replace / Comment Out :
code:
if (($searchMode == 'match-natural') || ($searchMode == 'both-match-natural')) {
  /* Natural language mode altijd default in MySQL 5.0 en 5.1, but cannot be explicitly defined in MySQL 5.0 */
  $queryPart = " MATCH(" . $field . ") AGAINST ('" . $this->_db->safe($searchValue) . "')";
  $filterValueSql[] = $queryPart;
} # if
With :
code:
$queryAdj = str_replace(array(' '), ' +', $searchValue);
if (($searchMode == 'match-natural') || ($searchMode == 'both-match-natural')) {
  $queryPart = " MATCH(" . $field . ") AGAINST ('" . $this->_db->safe($queryAdj) . "' IN BOOLEAN MODE)";
  $filterValueSql[] = $queryPart;
} # if

Nice work, unfortunately I installed PostegreSQL and it looks like the code is fairly dissimilar between MySQL and that. I found sorting by date makes it a little easier to read though.

# ? Dec 21, 2012 12:23

gary oldmans diary: Sep 26, 2005

Lusername posted:

But since it's decentralized, how susceptible is it to being 'poisoned' by outside organisations?

Without signing, I think it's wide open for poisoning. Not so much from NZB's with undesirable descriptions because someone could wind up posting an accurate description for the same set of files and then the undesirable spot would just go ignored and users wouldn't even know they pointed to the same files anyway. On the other hand, labeling undesirable files with desirable descriptions could waste time, waste bandwidth, and require db management for any tools people have organizing their downloads.
And it seems like it would be painfully easy to poison this system.

If spots had digital signatures it would only take the teensiest websites to keep track of spot contributors users have added to their whitelists and blacklists.

E: V Not sure if you're adding an unrelated thing or misunderstanding and arguing with me, but digital signatures would have nothing to do with security based on having an SA account.

gary oldmans diary fucked around with this message at 15:49 on Dec 21, 2012

# ? Dec 21, 2012 13:06

Telex: Feb 11, 2003

Hogburto posted:

And it seems like it would be painfully easy to poison this system.

If spots had digital signatures it would only take the teensiest websites to keep track of spot contributors users have added to their whitelists and blacklists.

And the obvious thing nobody will really admit, is that if it's easy for anyone with $10 to sign up here, it ain't like whatever system that shows up in the future will be unable to be infiltrated by a guy who is making money from the RIAA/MPAA/whoeverthefuckispayingtoscrewwithTV. If a guy is profiting, or worse yet a company of guys, then it'll be able to be infiltrated, poisoned and ultimately require an entirely different set of hurdles to come up with a sanitized solution if one is declared to be The Best way of doing it.

newznab is really the only way to do it right, since it only does what's necessary and that's scan a group and we all just hope that things that get posted are labeled correctly same as it's been in usenet before these indexers came around to make the lazy and "bad with computar" people able to do it and draw maybe a bit too much attention to the whole thing.

# ? Dec 21, 2012 15:31

Ronald Duck: Jun 26, 2005; My mum says I’m cool.

Hogburto posted:

Without signing, I think it's wide open for poisoning.

Certain NZB files are marked as coming from someone that's trusted. I'm not sure how the system works (Is the userlist downloaded from usenet or hardcoded into spotweb?), but there is some form of user authentication there.

And about spam:

quote:

Whenever spots are marked as spam from within any of the Spotnet clients, this happens by posting a message in the usenet groups. Spotweb is able to retrieve those spots as well and calculate a reputation score for the spot using this information. This score is displayed beside the spotlisting.

I guess if someone files fake spam reports the system would eventually realise and ignore them?

Edit: there's an external blacklist and whitelist: Config > Settings > Performace > Fetch external Whitelist and Fetch External Blacklist

Ronald Duck fucked around with this message at 15:58 on Dec 21, 2012

# ? Dec 21, 2012 15:53

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

For anyone who cares, as a test, a week ago I disabled all my providers in SickBeard except for my SpotWeb install.

All my stuff downloaded fine. As far as a delay between airing and downloading...it was minimal. No more delay than a couple hours AFAICT, and a lot of stuff downloaded just as soon as it was available on usenet.

I'm pretty sure someone is running some sort of newznab>spotnet script. Actually probably not even that. The regexes aren't too terrible to write or just rip out of newznab and incorporate into a custom script whose only purpose is to do just that.

Telex posted:

newznab is really the only way to do it right, since it only does what's necessary and that's scan a group and we all just hope that things that get posted are labeled correctly same as it's been in usenet before these indexers came around to make the lazy and "bad with computar" people able to do it and draw maybe a bit too much attention to the whole thing.

There's no difference between hoping people don't poison SpotWeb and hoping they don't poison usenet uploads.

Thermopyle fucked around with this message at 02:09 on Dec 22, 2012

# ? Dec 22, 2012 02:07

visuvius: Sep 24, 2007; sta da moor

So are you guys thinking sites like nzbmatrix and nzbs.org not really going to be coming back? I've gotten so drat used to the ease of nzbs.org that I haven't at all kept up to date with the new poo poo.

Going forward, is the only solution going to be a newznab/SpotWeb type setup you guys are working on?

# ? Dec 22, 2012 05:06

Vykk.Draygo: Jan 17, 2004; I say salesmen and women of the world unite!

visuvius posted:

So are you guys thinking sites like nzbmatrix and nzbs.org not really going to be coming back? I've gotten so drat used to the ease of nzbs.org that I haven't at all kept up to date with the new poo poo.

I feel like this has to be explained at least once a day but nzbs.org is not gone, you just can't get to it the normal way. Here's the link I had bookmarked before I just found out I was purged.

edit: oh yeah and they don't allow hotlinking so you have to copy and paste the URL or type it in yourself.

# ? Dec 22, 2012 05:13

visuvius: Sep 24, 2007; sta da moor

Vykk.Draygo posted:

I feel like this has to be explained at least once a day but nzbs.org is not gone, you just can't get to it the normal way. Here's the link I had bookmarked before I just found out I was purged.

edit: oh yeah and they don't allow hotlinking so you have to copy and paste the URL or type it in yourself.

I'll be damned. Thanks for that.

# ? Dec 22, 2012 05:22

an actual cat irl: Aug 29, 2004

I've just installed Spotweb to see how it shapes up compared to Newznab. So far, it seems pretty sweet; it's nice that it can run perfectly happily on a Linux VM with very little CPU load, as opposed to Newznab which has my VPS host sending me lovely warning emails on a daily basis about CPU usage and disk io.

One question - how does one actually contribute to Spotnet, in terms of both comments and spots? I can see existing comments on spots, but don't see any obvious way to add my own.

edit: sorry...ignore this question. I was logged in as the 'admin' user, and apparently these features aren't available with that account. Logging in as a regular user fixed it.

an actual cat irl fucked around with this message at 14:04 on Dec 22, 2012

# ? Dec 22, 2012 13:34

meatpimp: May 15, 2004; Psst -- Wanna buy

EVERYWHERE
some high-quality thread's DESTROYED!

Does anybody use Easynews? I've been with them for over a decade and they've been solid. Today, though, I got an email that I had a PM there, went to retrieve it, only to find the whole site down... :ohdear:

Nevermind, they just tweeted that they were being ddos'd.

meatpimp fucked around with this message at 16:44 on Dec 22, 2012

# ? Dec 22, 2012 15:51

TraderStav: May 19, 2006; It feels like I was standing my entire life and I just sat down

I'm having a hell of a time getting Spotweb to work in Windows. I did the easy install and after it installs, I install services but it seems that I have other services running on port 80 (Teamviewer). I change the port in custom.conf to something that should be free and use the restart server icon. It restarts the DB but the Apache fails to start with error code 1 or so.

Is anyone able to offer an idiots guide to getting this started, because I am one.

# ? Dec 22, 2012 16:11

flappin fish: Jul 4, 2005

meatpimp posted:

Does anybody use Easynews? I've been with them for over a decade and they've been solid. Today, though, I got an email that I had a PM there, went to retrieve it, only to find the whole site down...

Their Twitter says they're being DDOSed.

# ? Dec 22, 2012 16:37

mcsquared: Nov 19, 2005

Hey I got spotweb running in a VM. Cool. How long is the initial retrieve going to take on a terrible 1.5Mbps connection? We're about to leave for the holidays so... should I just leave this thing running if its going to take days?

Just need to decide whether or not to verwijder de erotiek spots

# ? Dec 22, 2012 17:27

an actual cat irl: Aug 29, 2004

mcsquared posted:

Hey I got spotweb running in a VM. Cool. How long is the initial retrieve going to take on a terrible 1.5Mbps connection? We're about to leave for the holidays so... should I just leave this thing running if its going to take days?

Just need to decide whether or not to verwijder de erotiek spots

My adsl runs at about 13mbps, and I managed to download all the spots in about 15-20 mins. It then took about another 20 mins to download all the comments. So, even on a slow DSL connection, I wouldn't expect it to take more than a few hours.

# ? Dec 22, 2012 18:08

The Gunslinger: Jul 24, 2004; Do not forget the face of your father.; Fun Shoe

Decided to be lazy and use Spotweb Easy, very well automated for the most part. Up and running as a SB/CP source in 5 minutes. I just hope I don't end up with hard coded dutch subs on everything or something silly.

# ? Dec 22, 2012 19:25

mcsquared: Nov 19, 2005

moron posted:

My adsl runs at about 13mbps, and I managed to download all the spots in about 15-20 mins. It then took about another 20 mins to download all the comments. So, even on a slow DSL connection, I wouldn't expect it to take more than a few hours.

Thanks! It's moved on to the comments so it'll be done before I go.

For anyone dumb like me, when configuring it to send to sabnzbd, put in just the root address (i.e. http://127.0.0.1:8080) and not http://127.0.0.1:8080/sabnzbd/ like I did and then spend 20 minutes wondering why it aint working but throwing no errors.

# ? Dec 22, 2012 19:32

Adbot: ADBOT LOVES YOU

# ? Apr 25, 2024 19:59

Ronald Duck: Jun 26, 2005; My mum says I’m cool.

moron posted:

edit: sorry...ignore this question. I was logged in as the 'admin' user, and apparently these features aren't available with that account. Logging in as a regular user fixed it.

I was confused by that for a while as well :downs:

. The default Admin account can't post Spots/Comments or download new spots.

If anyone plans on opening their site publicly make sure you change the default Admin password as well (it's set to admin).

TraderStav posted:

I'm having a hell of a time getting Spotweb to work in Windows. I did the easy install and after it installs, I install services but it seems that I have other services running on port 80 (Teamviewer). I change the port in custom.conf to something that should be free and use the restart server icon. It restarts the DB but the Apache fails to start with error code 1 or so.

Is anyone able to offer an idiots guide to getting this started, because I am one.

Open your start menu. Spotweb Easy > Management > Apache2 Port

In that file all you need to change is List and (optionally) ServerName. To change from port 80 to 90, go from:

code:

## httpd
Listen 80
ServerName localhost:80

to:

code:

## httpd
Listen 90
ServerName localhost:90

And then restart the server. (Start Menu > Spotweb Easy > Management > Restart Server)

Make sure you didn't edit URL: line (keep the # in front of it), that's just telling you what your new URL would look like. For port 90 it would be http://localhost:90/spotweb

Ronald Duck fucked around with this message at 20:41 on Dec 22, 2012

# ? Dec 22, 2012 20:22

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Usenet Megathread - No invite discussion!

«‹›357 »