|
Ugh, Sickbeard keeps grabbing the proper for a file and corrupting it (the original file is ~360mb, after it pulls the proper it's 11mb). Do I have to have the post-processing for sabnzbd set to Download for Sickbeard to handle it properly? It's on +unpack at the moment.
|
#
¿
May 14, 2011 07:50
|
|
|
|
| # ¿ Apr 23, 2024 23:54 |
|
|
Ugh, windows permissions issues ahoy. Basically, SB/CP download stuff into their temporary folder and then rename/move it to individual folders across a couple of different drives (not a problem here) - those folders are then shared across the network and imported by xbmc as sources. My problem is that I need to have the security on new files set to allow read for Everyone (otherwise the networked xbmc pc can't play it). Anyone else come across this? I have the security options on both the shared folders and temporary download folders set to allow read for Everyone, but the individual files being downloaded don't get that set, so while the folder is shared, the file itself can't be accessed.
|
|
#
¿
Jun 14, 2011 19:25
|
|
|
Kachunkachunk posted:I used to run with a Windows host like that, but I actually created additional users that I wanted people to log in as, to access shares (didn't use Everyone). This allowed some users to write in shared directories and some others to not. I'd suggest trying that with your "XBMC user" (do this for the share and the NTFS file system directories, and re-add your shares) and it should work fine. Security's not particularly an issue for this box, so I might be lazy and try the latter. Cheers.
|
|
#
¿
Jun 15, 2011 07:51
|
|
|
Running my own indexer and boy howdy does it take a long time to index everything 
|
|
#
¿
Jun 29, 2011 10:53
|
|
|
I wrote a python-based Newznab replacement, since it was giving me a massive loving headache: https://github.com/Murodese/pynab It's purely API-oriented - there's no front-end and I'm not going to write one, since the front-end on my current indexer never gets used. No post-processing yet either, so Sickbeardcan only use its RSS search currently (which it mostly does, anyway). CouchPotato should mostly use it fine. Postproc will take me the weekend or so to complete. Of note, bulk importing NZBs is hilariously faster with this than Newznab - a few hours to import the 60ish gb NZB dump from mid-2012 compared to a few weeks/months with NN+. Takes much less time to process parts, binaries and releases, too. Pretty much requires a NN+ ID to get regexes, though. I only wrote the install scripts tonight, so if you try it out let me know how it goes - it could be horribly broken, but the limited testing I did seemed ok. (I even tried to write it with a really nice OO-design and it completely loving failed because it increased processing time some 50,000x over doing it the quick-but-ugly way)
|
|
#
¿
Oct 10, 2013 14:34
|
|
|
Thermopyle posted:Ha, I've been working on one in Python in my free time as well, because newznab is a trainwreck. NN's code is truly horrifying. If you want to see something even worse, check out nZEDb's lib/category.php 
|
|
#
¿
Oct 10, 2013 16:05
|
|
|
I don't remember if I ever talked about this on here, so here you go (also sorry Thermopyle, I only just saw your post from April 2013 today, I do not read the thread regularly  ):quote:It's been a long-rear end time since this project was first on here and it's changed fairly significantly since then, so I figured I'd re-mention it for anyone that's after a simple indexer. Double tl;dr: a really fast usenet indexer. It's pretty stable now, being in use by quite a few people for a couple of years, and most of the bugs are hammered out. If you're looking for something to index weird groups not covered by other indexers or want to just run your own indexer from home, this works. In the future I'll package it into a tiny portable thing that can just provide people with their own poo poo, without having to pay for indexer access or send traffic externally (for people that live in Australia, for instance). Totally looking for people to build more pre-db integration into it, as well. We do a couple of groups, but there are some big ones missing that I don't have any kind of pre access to.
|
|
#
¿
Apr 13, 2015 04:41
|
|
|
Mr Chips posted:Request: add a database abstraction layer so I can point it at oracle instead of postgresql There is one, but parts are heavily optimized and I have to deal with certain bits in a DBMS specific way. Mysql support exists but isn't fast, for instance. e; actually, you can try it just by changing the db engine in your config to 'cx_oracle' and pip3 install cx_oracle. It'll be slow as poo poo saving segments to the db, but that's about the same speed as newznab works 
Murodese fucked around with this message at 05:52 on Apr 13, 2015 |
|
#
¿
Apr 13, 2015 05:18
|
|
|
Yeah, chuck it to me when it's done, I'd love to see it. I was originally going to write this to use distributed collection/processing, but you can already do that to some extent by using the DB as a centralised event handler and just split the various responsibilities over different machines.
|
|
#
¿
Apr 13, 2015 07:18
|
|
|
|
| # ¿ Apr 23, 2024 23:54 |
|
|
xgalaxy posted:So what are the storage and bandwidth implications if someone wanted to run their own indexer for their own personal use? I index something like 25 popular groups (barring some really spammy ones), and have backfilled to around ~700-900 days on stuff. End result is that I have around 520,000 releases. My DB looks like this (I've noted the tables that fluctuate really wildly as you're indexing, because the data is collected and then aggregated, compressed and deleted): pre:nspname | relname | size | refrelname | relidxrefrelname | relfilenode | relkind | reltuples | relpages ----------+------------------------------------+------------+-------------------------+-------------------------+-------------+---------+-----------+---------- pg_toast | pg_toast_4704468 | 12 GB | nzbs | | 4704472 | t | 4866792 | 1579537 public | segments | 5790 MB | pg_toast_4704738 | | 5192408 | r | 4761148 | 741172 <-- fluctuates wildly public | ix_segments_segment | 2012 MB | | | 5192415 | i | 4761148 | 257486 <-- same with this one public | ix_segments_part_id | 1599 MB | | | 5192416 | i | 4761148 | 204694 <-- and this one public | segments_pkey | 1288 MB | | | 5192414 | i | 4761148 | 164855 <-- ..and this one public | nzbs | 287 MB | pg_toast_4704468 | | 4704468 | r | 494241 | 36707 public | pres | 234 MB | pg_toast_4704419 | | 4704419 | r | 1026911 | 29956 public | releases | 168 MB | pg_toast_4704633 | | 5581407 | r | 519001 | 21510 pg_toast | pg_toast_4704468_index | 158 MB | pg_toast_4704468 | nzbs | 4704474 | i | 5191460 | 20280 public | ix_pres_name | 116 MB | | | 4704430 | i | 1026911 | 14824 public | nfos | 105 MB | pg_toast_4704457 | | 5581477 | r | 138937 | 13496 pg_toast | pg_toast_4704457 | 85 MB | nfos | | 5581480 | t | 57122 | 10881 public | ix_pres_requestgroup | 59 MB | | | 4854331 | i | 1026911 | 7580 public | pres_uniq | 58 MB | | | 5223660 | i | 1026911 | 7396 public | releases_uniq | 56 MB | | | 5581431 | i | 519001 | 7158 public | ix_parts_group_name | 53 MB | | | 5192401 | i | 13987 | 6777 public | ix_releases_search_name | 52 MB | | | 5581428 | i | 519001 | 6597 public | ix_parts_binary_id | 36 MB | | | 5192403 | i | 13987 | 4599 public | ix_parts_posted | 36 MB | | | 5192404 | i | 13987 | 4587 public | ix_pres_requestid | 34 MB | | | 4854332 | i | 1026911 | 4321 public | ix_parts_total_segments | 34 MB | | | 5192402 | i | 13987 | 4294 public | parts_pkey | 33 MB | | | 5192400 | i | 13987 | 4287 My network usage while backfilling, updating and post-processing looks like this: pre:Device em1 [10.1.1.200] (1/2): =========================================================================================================================================================================== Incoming: | ##|### # |##| # ### .##| # #| # ###### ## |####| .#. ### #### .#. ## ##| # ####### ## ######. ### .### .#### ###.##. .###. # #######. |### #######.### ####. ###### ####### ##### ## . .######## #### ############. .##### ############## ##### ## ||# ######### ##### ##############|#######|############## ###### Curr: 3.22 MBit/s ## |.|###. .######### ###### .. .###################################### .####### Avg: 2.35 MBit/s ## #.|####### ########## |############# ################################################# Min: 552.00 Bit/s ############# .########### .|##############| .################################################## Max: 11.08 MBit/s #############. .##############################...#################################################### Ttl: 196.72 MByte
|
|
#
¿
Apr 15, 2015 04:21
|
|