/tnt/ - Tournaments & Events

This board is for hosting tournaments and other organized competitions, be it either events, contests, or anything where a winner must be determined through votes or otherwise. Just for this board, image duplicates are enabled and the bump limits are set extra high. Roleplaying is encouraged, unless event hosts ask otherwise.

Please consider donating! Thanks!

Thread stats: 61 posts, 13 files (13 image(s))

Toggle poster info Replying to /tnt/12579 Close window
save file
image:166597135792.png(1MB , 1200x1200 , beepboop.png)
With the way things have gone in ms. /co/ it's clear theres a lot of junk in hosting that could benefit from software automation. This thread will be the place to discuss such themes and/or contribute to related projects.
Of course since I'm not a lazy faggot I also went ahead and dipped my toes into this subject, even if I have no background in programming besides picking up a book once in a while about the theme.
Still I've managed to begin this(somewhat) functional project:
Probably looks horrible to the experienced eyes, but its functional. With what has been written so far one can change the settings in the main Archive-bot file and it'll do as it says, grabbing the first valid thread that fits the pattern, reading every single post looking for valid matches fitting the parameters and saving it all in a text file.
Default parameters are as follows:
thread_pattern = "mr-co"
board = "co"
sleep_minutes = 5
minreplies = 8
(the above can easily vary)
Post must have an image attached, at least 1 reply, more replies than minreplies setting and have a string that fits the search pattern of beginning with 'Nominating/nominating', followed by a 'from' and ending with a dot(.). If it doesn't fit one of these it's immediatelly thrown out.
If it passes its saved on a text file as "x( y)" where x is all between 'nominating' and 'from', and y is all between 'from' and the dot.
I've experienced some bugs which *should* be patched by now in the git files and still havent tested a long time run of it yet. It's also missing a feature to remove similar entries, which I'm planning on using difflib for.
Added the function to clean up repeats. Seems to miss when there are several repeats but with multiple runs on the file they all get removed.
This sounds amazing.
Added up a library to deal with updating the google spreadsheet. Since it requires some setting up with the credentials I'll update the readme with a link to a python book explaining it.
It should now work and do the things I had planned it to do, assuming there's no sneaky bugs(there probably are, shit hasn't been tested during an actual event). Now just gotta make it a bit more user-friendly and wip out a pre-compiled exe to leave there.
Thanks! I was kinda surprised no one else seemed to have done something like this, given how long the tourneys have been going.
Seconding this. I remember the topic of automating the nomination process coming up but I never thought anyone would actually take the initiative to do it. This is definitely something that could improve future tournaments. Very impressed.
This is gonna be hella useful since nominations keep rising every year. Just two questions: will the robot have a name? Will it be a host too?
>will the robot have a name?
Yes. Currently the github project name is more of a placeholder. I'm thinking of going with simply 'Jenny' since it was suggested during some talks in the ms /co/ thread that discussed automation, and it fits.
But if you have other suggestions, I'm open to it.
>Will it be a host too?
As in the bot also creating the threads? If so, no, there's no current plans for that. Mainly because from what I see that sounds like something that would get the mods pissed.
Tho I do know they're lenient if the one doing it uses a pass(some youtuber bought one, made an AI-text bot and had it make hundreds of posts and they didnt give a shit), that would be an uneeded expence for the one hosting it.
Also smaller but relevant point to this, drawfags feel more a compliment if their drawing becomes an OP by being chosen by a human, than if a bot did it by choosing randomly.
personally I'd call it J3nny just to differentiate from Jenny plus leet speak
also probably can't be a host but a judge for sure
Should it be used it ought to have a mention in the OPs for sure. If nothing else so anons dont immediatelly assume a fuck-up made by the bot was done by the host.
Small update, minor code shit, mostly text changes.
New repo name aswell.
Big thing added is the "j3nny.rar" file which contains a pyinstaller output for the bot, download it and follow the steps in the project README, no need to downloaded python or libraries, pyinstaller took care of that shit.
I have confidence in its current state, but its still worth being skeptical of how it'll do in a real nomination event, as it hasnt yet been tested on one.
Oh yeah, also the exe icon is just a crop of jenny's face I got from the first google result, can't be arsed getting anything prettier.
Find a nicer picture for the icon, or get some drawfag to do it, and I'll update it.
I will be testing this for Queen. Thank you for the update.
Update: NSA told me he isn't using J3nny yet.
Spideranon !!hAaBXjbZBz7
Seeing that I foolishly let a bug fly by that ruined its output, its for the best. Thankfully I dont have much to do tomorrow so I can test it on my own.

Anyone who downloaded the .rar, feel free to delete it. Even with the google credentials, you can download those again from google. And the .pickle files generated from it are temporary(as i've come to learn today) and could mess shit if left there to rot.
Can you test it with Queen /v/ and share the results? I wanted to use it. But since I didn't have experience with it I couldn't utilize it well.
Spideranon !!hAaBXjbZBz7
What did you have trouble with?
And sure, I've been testing with Queen /v/, current results:
Could've gotten more but a few times here and there I had to stop it to fix bugs, or delete some data and such.
Gonna just let it run and do its thing until nominations are over now, then I'll look over to see what I need to fix.
The same issues you had. I stopped using it for this year. Can you share the results for your doc? It's restricted. Thank you for doing this.
Spideranon !!hAaBXjbZBz7
Right, right. Set it to public. Doc only has the results for the last thread, I was doing some tweaks to make the output be a bit cleaner, so for speed's sake I deleted previous data.
Some of the "messier" posts seem to have skipped by the bot, but generally ones consisting of "nominating x from y" are always scraped and converted correctly.

By the way, assuming you gave it a trial before, were you building from the python files or the .rar?
I will pray everyday for you to die a slow and agonizing death
Sooo why don't you communicate more directly with the thread amid all shit you stirred? Clearly you followed it and you've seen the vast majority haven't agreed with you. In all you're serving the board, not your interpretation of a proper tournament (which isn't consistent anyways).
I was using the rar. I had issues since it would only get the nominating soandso from etc. A lot of anons were nominating by just saying the name and adding flavor text then the series. It threw it off so I just quit it. At that point I just wanted to keep up. I'll try it again after the Qualifiers. I'm not a programmer so I can't troubleshoot like you. Your list is longer. Some of the titles are off but really good. Just needs manual touches, but overall that's a lot better than collecting them by hand.
Spideranon !!hAaBXjbZBz7
I see. Originally I had made its analysis process strict to avoid false positives, but then that lead to several false negatives in this test. Now its been changed so that if the post has an image, above minimum replies and isn't too large(current top text size is 100 characters) it'll try to make it fit the "someone(some series)" clean up, otherwise adding the post as is.
Will lead to some false positives, specially if anons are made aware their posts are being scraped by the bot, but that'll be left for the human host to deal with.

Github already updated with the changes. Rar to-be-done at a later time, because one of the libraries(pygsheets) seems to be causing some troubles with the library I use to make the standalone .exe(pyinstaller).
Spideranon !!hAaBXjbZBz7
Changed the compiler to nuitka and fixed the problem it and pyinstaller were having with pygsheets. Seems some code needed to change in some files, and some stuff wasnt getting copied.
Release of the v1.0(XJ-1.0) on the github.
Are we using it for King of /v/? If yes, don't forget to put J3nny's name in the list of judges.
Spideranon !!hAaBXjbZBz7
>Are we using it for King of /v/?
Up for whoever will be hosting it to decide. I am not and have never been a host.
I had to fix up my computer. It took a good but for me to install J3nny the first time, but since my computer was wiped I can't use it. Can you please create a public doc using it with King /v/? I'll credit you and J3nny.
Friendly reminder that J3nny, if you use her, should be credited as a judge.
If Spideranon is here then yes. I can't figure out how to get it to work again. My brain fizzled out during installation the first time.
Spideranon !!hAaBXjbZBz7
Been busy and away from computer so no can do, will probably only manage to run it, if at all during the tail end of nominations. What was your intall issue, should be easy to do. Just download the newest release from github(the .rar) and run j3nny.exe. Had trouble finding the link, downloading or something else?
Also i will be clear, if you do manage to run it now BACKUP THE SHEET, as the bot clears the sheet its given and then updates it with info from nominations.txt, which will be empty on a first run
save file
image:167736519932.png(41kB , 550x350 , Figure_1.png)
Still not done with the scrapebot project. Plan is to make it so it scans the catalog for the thread, then stays on thread adding new entries until the thread dies, sleeping less the further down the catalog the thread is. Then it uses archive search(currently the only method) should it not find a new thread.
Furthermore it should shut itself down once the time for nominations is done, probably via getting info from an https://itsalmo.st/ link since afaik thats the default used for timing here. This time I'll only update the github/release a compiled version once its been tested on a contest proper, since last time a bug I missed made it give junk output.

Another point I see automation could come in handy, vote analysis. If my memory is correct, amid the drama in one of last year's tourneys, NSA gave some sheets with vote data for anons to verify on their own. However such raw data is a handful to digest.
A script to automatically quantify, analyze and graph it could solve that problem and give more fidelity to the votes. Pic related, data obviously just random shit I graphed and correlated(via Pearson's formula from numpy).
...can't find the post with the file he gave though, if someone could get me that that would be wonderful.

A program that gives a real time graph view of votes could also be made for use by hosts... if the forms give that info, idk if it does. I've also been thinking of the math that could be used to flag fraudulous votes or results. I considered Benford's law graph combined with a chi-squared test, but I *think* the sample size for contests are generally too small for Benford's to be accurate, someone correct me if I'm wrong. Possibly just going with the more basic probability and statistical tests on the data.
Found it
Spideranon !!hAaBXjbZBz7
Oh it was of the famed cheated round too, excellent. I'll play around with it.
Thanks for the help.
save file
image:167762587167.png(204kB , 1366x664 , Figure_1.png)
Hmm, in hindsight, plotting so many values turned the graph into a bit of a mess, more work may be needed to make them pretty. And in hindsight again I dont *really* know which may be useful for proving or rejecting a cheating hypothesis.
Maybe if there was a "trustworthy" sheet of votes to compare with, but dunno if NSA gave such a thing
save file
image:167762588689.png(177kB , 1366x664 , Figure_2.png)
save file
image:167762597643.png(16kB , 640x480 , Figure_4.png)
Big amount of votes are usually made right after polls are opened, kind of expected.
save file
image:167762643912.png(67kB , 1366x664 , Benford.png)
Decided back on this and ran a Benford's law on ALL votes. I thought samples may be small at first but I foolishly thought of running on a per character basis. With all data in the matrix as sample(24 hours times 32 characters = 768) the sample size is more than enough for Benford.
Graph came out anomalous, but I'm still not sure on this test. Maybe this is a result of the great amount of votes on hour 1 as shown above. Maybe I should play around with sampling and if done on a minute by minute it'd come out diferent.
And again, without a trustworthy sheet to compare to, I'm iffy about interpreting it.
save file
image:167762668358.jpg(724kB , 1953x1242 , C5BEB029-07EF-4E42-AD8E-8048F9A50710.jpeg)
From what I’m getting from a voting perspective is that the first five hours and the last four hours are the most voters come to vote and are the most important time periods for campaigners to campaign for their characters which is quite insightful.

Tho, those spikes if connected to a certain group of characters can help us distinguish cheaters in these tournaments. Just a personal thought.
save file
image:167788164661.png(139kB , 2512x628 , Figure_3.png)
If they are I cant particularly tell based on just the graphs. But maybe its just my interpretation, heres the heatmap of characters votes per hour and the sheet its mapped to for any would be interested.
If NSA had shared a 'non cheated' graph interpretation would be easier, cant tell what is and isnt an anomaly without something to compare with.
Small addendum, the X axis is wrong, values are actually sum of character votes instead of votes, shouldve been divided by 16(32 characters divided by 2 to find n of matches), bar proportions still apply either way.
Spideranon !!hAaBXjbZBz7

Update, now it looks for a valid thread in the catalog first and stays on scraping it until it dies, then falls back to the previous archive search and repeat until the end time(given by an almo.st lnk) is reached.
Likely final version, not much else planned but fixing stuff.
Only other thing going through my mind was using AI models for the Named Entity Recognition task to grab the names of characters and media. Still need to measure the cost benefit of that tho, distilled models can fail on more unorthodox character/media names but larger models are, obviously, larger.
So this bot is now capable of collecting nominations itself without someone needing to go one by one?
Spideranon !!hAaBXjbZBz7
Always was, but the previous method was dumber and required more handholding of whoever was using it. Now once inputs are given it can be let to do its thing and it'll collect nominations and stop itself when it must instead of going on forever.

Correctly gathering the nomination for posts that dont follow a predictable pattern is still something it cant do and requires user intervention, but I'm looking at NER AI models to see if that can be fixed.

Also worth noting a new release hasnt been compiled yet, just updated the git code itself.
Is it possible to get in contact with you outside of plus4chan?
Spideranon !!hAaBXjbZBz7
Here's a throwaway gmail
Spideranon !!hAaBXjbZBz7
New, and hopefully last, release of the nomination bot. Fixed some stuff, added some error handling for stuff I could reasonably predict, blacklist and time handling.


Decided to not put in any NER, bloated it too much for not much of an upgrade imo. If the bot puts in some invalid or bad entry its then up to the host to fix it in nominations.txt while the bot is sleeping.
Gonna still have to see about prettifying the graph program since as it stands its a mess since theres so many characters to graph and Im doing it in the bare bones way, and its user unfriendly as shit. Gonna need a GUI, which may be a pain to make.
>(Standalone built with Pyinstaller, Windows only)
...does Windows XP count?
save file
image:168954179828.png(41kB , 639x692 , it just works.png)
Im not much of an UI autist. Is this understandable enough?
Pretend the bottom graph is actually vote relevant, was just trying shit out.
Havent tested but I'm almost certain it doesnt, XP's firefox version is too old for one of the modules I used.
Have tested and can garantee it works on Win 7 64bits and above. Standalone was built on Win 10 64bits.
Shot you an email.
save file
image:169223507331.png(1.04MB , 750x750 , old radio.png)
No AI's taking this radio for a roll, I promise you that.
I've been testing the J3nny. While I can get it to start, I don't receive any results and it doesn't generate a sheet. I tried swapping the URL to /tnt/ to test it, but I can't get it to work.
I"ve just been messing with the thread parcer.py. It may be because the threads I used were deleted from 4channel, and it's not originally configured for /tnt/. I just want to make sure I have this right. Is there a way for me to test it?
Spideranon !!hAaBXjbZBz7
Does it still generate a question prompting for input or close without showing anything?
If it does, are you inputting the correct types of inputs? If it gets to the part asking for the almo.st timer, do you give it a link to one that isnt expired?
>the threads I used were deleted from 4channel
Deleted threads lead to a 404 page, which is why I use the function is404 to check if they were pruned. There's nothing to scrape there, you'd have to scrape an archive, which I did not script j3nny to do.
>it's not originally configured for /tnt/
Very much, I cannot garantee it will work on anything but 4channel. It could theoretically work on imageboards with an identical HTML however, which I do not know if plus4chan has.
>Is there a way for me to test it?
It as in j3nny? Just get the latest release(j3nny.7zp) on the top left side of the github page.
Sent you another email.
Was J3nny even used in a tournament?
Spideranon !!hAaBXjbZBz7
Not before this one, no.
Why wasn't she listed as a judge?
Spideranon !!hAaBXjbZBz7

Project updated, new release.
To wager a guess, because NSA couldnt get it running. Turns out a library was causing issues with the standalone even if not with the script itself, which is what I ran. That bit was swapped.
save file
image:169974556487.jpg(78kB , 758x652 , b80aafd4c89ebd1.jpg)
Fixed the error on the latest release. Since basically every time now something went to shit I figure I'll only make a release when the script has been tested on an actual tourney. Or a slap a huge ass "trial" on the title if not.
Also instead of a whole folder now they'll be just standalone .exes, not sure why I didnt do that before.

Vote sheet graph script still not in what I'd call a releaseable state. Making UIs kills me inside and so does looking back on this spaghetti mess of code.
Plus still not sure on the methods to use, even Benford's has its caveates.

Small blogpost: I'm set to get a job in the tech industry. Here's your remainder that dedication to autistic passion projects can add up to something better over time. We're all gonna make it bros.
>Still not listed as a judge.
J3nny should be a userscript
save file
image:170622291426.png(150kB , 627x2079 , test results.png)
>fooling around with AI models
>try doing Named Entity Recognition on tourney data again
>run first test
>0.62 accuracy, basically a shitty guess
>run a multilanguage dataset
>spikes to 0.95
Zamn, in hindsight I guess it was because the names are quite diverse in language( data from Queen /v/) so a basic english model wasn't cutting it.
Still need to figure out if it can do well with both character name and series names consistently too, and if so how/if it classifies them well, but this could be something
Program size/cpu increase not considered much since this particular library(spacy) is nice enough to let me trim the fat of using anything BUT the NER model
For a program that's used at most like 4 times a year that sounds like a waste of time
save file
image:170752302173.jpg(100kB , 768x894 , oh-no-anyway.jpg)
>in addition to the url change the cloudflare anti-bot thing was messing with the library I was using for requests
Oh no
>change to diferent library
Anyway, fix release: https://github.com/ZamboLambo/J3nny/releases/tag/1.3.0

Probably last one for a long while since I'm planning to change a ton of shit now that I've taken the OOP pill.