|
im making a web toy that makes a youtube web player using all the vids in a thread, decided to make this thread to post updates about it instead of making GBS threads up the meta thread so far ive managed to make new endpoints for the unofficial somethingawful API (check it out) for viewing the posts in threads, so that i dont have to write an entire web scraper for the project and can just make an API call thankfully the youtube API seems slightly less useless than i first anticipated so getting the actual player to work shouldnt be TOO horrible
|
#
?
Jan 17, 2022 12:31
|
|
|
|
| # ? Apr 25, 2024 14:20 |
|
|
should mention that the endpoint for viewing full threads absolutely shits itself if the thread is too long (tested it on the cjs thread in yospos and it took so long that cloudflare thought it died) so im probably going to add a version for making playlists out of specific pages and/or page ranges for if you want to grab poo poo from a particularly long thread but first im going to focus on the Main Thing, and do those two things after thats up and running
|
|
#
?
Jan 17, 2022 12:43
|
|
|
have got an incredibly simple player up and running but now i need to wrestle with jinja to see how i can import this loving array into js from my flask backend
|
|
#
?
Jan 17, 2022 16:39
|
|
|
thanks for the updates hbag. looking forward to seeing this in action
|
|
#
?
Jan 17, 2022 17:05
|
|
|
i actually have literally the entire player working but now i just need to add a shuffle function and also make it look nicer than uh this 
|
|
#
?
Jan 17, 2022 17:07
|
|
|
 seems good enough for now i noticed that a lot of videos seem to not work with the iframe API so they display as "not available" and get skipped, so rip lmao
|
|
#
?
Jan 17, 2022 17:56
|
|
|
drat. thats loving sweet. God bless you
|
|
#
?
Jan 17, 2022 18:11
|
|
|
seems the API can handle, at most, 5 pages at a time so im probably going to make a new endpoint that just gets the last 5 pages of a thread and use THAT for the playlist generator
|
|
#
?
Jan 18, 2022 02:59
|
|
|
lol nvm turns out i can use async functions instead of waiting for each individual page like a psycho
|
|
#
?
Jan 18, 2022 09:50
|
|
|
async functions are proving to be dumb and lame so i might just add a function to the site where you can start monitoring a thread and itll build the playlist from a list of links stored in a db or some poo poo
|
|
#
?
Jan 19, 2022 00:52
|
|
|
im not much of a computer guy and i have no clue how you set this up on your end. but wouldn't a good way to cut down on scraping and performance problems be to just archive the results and show a list by thread name or some poo poo. then only run a scrape when someone enters a threadid, either to add a new one or update and existing one. this would also make it easier to use because it's kind of a pita on my phone. and this data seems like it should be easy to store as a table or text file of some kind so it's not like it would eat up a bunch of your server space. edit I guess that's kind of exactly what you said in your last post lol
|
|
#
?
Jan 19, 2022 13:30
|
|
|
Larry Parrish posted:im not much of a computer guy and i have no clue how you set this up on your end. but wouldn't a good way to cut down on scraping and performance problems be to just archive the results and show a list by thread name or some poo poo. then only run a scrape when someone enters a threadid, either to add a new one or update and existing one. this would also make it easier to use because it's kind of a pita on my phone. and this data seems like it should be easy to store as a table or text file of some kind so it's not like it would eat up a bunch of your server space. yeah lmao kinda already up and running - it doesnt scrape the thread when you search it unless its a brand new thread. if the thread has been scraped before, it'll just load the results from a database and use that
|
|
#
?
Jan 19, 2022 13:36
|
|
|
u should make an index so my lazy rear end can just click link 
|
|
#
?
Jan 19, 2022 13:47
|
|
|
Larry Parrish posted:u should make an index so my lazy rear end can just click link yeah ill probably do something like that once ive made sure everything's working alright and not doxing me or something
|
|
#
?
Jan 19, 2022 13:50
|
|
|
gotmore ideas. scribbling them down for later. loving Diabolical
|
|
#
?
Jan 19, 2022 15:52
|
|
|
also lmao at this loving janky-rear end function for retrieving the playlists
hbag fucked around with this message at 21:52 on Dec 9, 2023 |
|
#
?
Jan 20, 2022 23:16
|
|
|
|
| # ? Apr 25, 2024 14:20 |
|
|
the thread scraper that the playlist generator uses has a few bugs that cause it to crash, so im gonna be working on fixing those letting you know because it means new playlists arent gonna be generated beyond the first page for now
|
|
#
?
Feb 24, 2022 02:58
|
|