Instadev
MySQL & C++ Developer
banned
LEVEL 2
90 XP
Hey premiums, just wanted to share something I will be working on real soon. My initial plan was to implement an active anticheat on my 2022 source like here (this is taken from my 2020 source):
I wanted to add a self-learning algorithm that learns if a cheater is an actual cheater based on GM feedback (or action taken against the individual) and to add a scoring system that is actually (at later stages) capable of determining if a player's score is high enough - take automatic action, thereby removing the need for GMs to BAN players, and only results in appeals handling.
That being said, I was introduced to something new called Redis, which does pretty much what I wanted to make from 0 in terms of replication layer functionality. I'll explain:
The server is running on your machine, and upon every restart, it loads all the data over and over again.
The server caches everything you do onto its memory (this is why the server is capable of remaining online even if you delete a whole table or I dare even say database) and is re-inserting data once every 90 seconds (or something around there). That's great but can still cause a slight loss.
I say why lose anything at any point?
This is exactly why I wanted to start working on something called a REPLICATION LAYER.
I do need to note: that's a lot of work and it will take a significant amount of time to accomplish, but once it's done it will be an AMAZING system to have, something many others will want as well.
Basically what it does is this: when the server initially loads all of its essential assets (as in, worldserver boot), it will also cache its data into a 3rd party software called Redis and into its memory alongside it. Great, now we have ram duplicates that are 100% matching and identical to each other. However, instead of saving stuff into the database, it will save it in the replication layer, which will handle the rest without causing any lag in-game.
Now let's assume the server crashes. Huge deal, a big mess, and lost data. Well... not really. The server's exact instance is located in the replication layer. THEORETICALLY, nobody lost anything. The server's being stored on the machine's RAM which is the fastest accessible memory, so loading any server will be SIGNIFICANTLY faster & make a full recovery no questions asked.
The fun part is, the replication layer can basically never crash or be overridden with needless data. It will handle all the MySQL insertions by itself and will do its absolute best to maintain high stability and rapid recovery in cases stability isn't an option, say - test server.
If the replication layer itself flushes data, the worst-case scenario is us going back to what we're used to right now, which is loading data that is 90s delayed.
The BIGGEST accomplishment of this system is the ability to make it so dynamic, you can literally run an instance of your server from any machine, as in - a fallback server.
You can very simply hand a "repack" of your server to a developer and he will be able to run the same instance of your server on his local machine without ever needing to download or even accessing the database.
This will require me to rewrite almost all the commands to support the replication layer, which as I've said is a HUGE project by itself. For now, the focus is to narrow the startup time to mere milliseconds and see if I'm actually able to restart the server and still maintain the exact same data on a repetitive basis.
As of right now it's all theoretical and has not been implemented in-game. I kindly ask you not to close this thread as I will post here the progress being made on my development machine for documentation purposes.
Feel free to ask questions and share your concerns, I will base my implementation of this system based on this. And no, as of right now it is not in a public repository and will not be at least until I implement it thoroughly, which can take days or even weeks.
Best of luck to me, lol
I wanted to add a self-learning algorithm that learns if a cheater is an actual cheater based on GM feedback (or action taken against the individual) and to add a scoring system that is actually (at later stages) capable of determining if a player's score is high enough - take automatic action, thereby removing the need for GMs to BAN players, and only results in appeals handling.
That being said, I was introduced to something new called Redis, which does pretty much what I wanted to make from 0 in terms of replication layer functionality. I'll explain:
The server is running on your machine, and upon every restart, it loads all the data over and over again.
The server caches everything you do onto its memory (this is why the server is capable of remaining online even if you delete a whole table or I dare even say database) and is re-inserting data once every 90 seconds (or something around there). That's great but can still cause a slight loss.
I say why lose anything at any point?
This is exactly why I wanted to start working on something called a REPLICATION LAYER.
I do need to note: that's a lot of work and it will take a significant amount of time to accomplish, but once it's done it will be an AMAZING system to have, something many others will want as well.
Basically what it does is this: when the server initially loads all of its essential assets (as in, worldserver boot), it will also cache its data into a 3rd party software called Redis and into its memory alongside it. Great, now we have ram duplicates that are 100% matching and identical to each other. However, instead of saving stuff into the database, it will save it in the replication layer, which will handle the rest without causing any lag in-game.
Now let's assume the server crashes. Huge deal, a big mess, and lost data. Well... not really. The server's exact instance is located in the replication layer. THEORETICALLY, nobody lost anything. The server's being stored on the machine's RAM which is the fastest accessible memory, so loading any server will be SIGNIFICANTLY faster & make a full recovery no questions asked.
The fun part is, the replication layer can basically never crash or be overridden with needless data. It will handle all the MySQL insertions by itself and will do its absolute best to maintain high stability and rapid recovery in cases stability isn't an option, say - test server.
If the replication layer itself flushes data, the worst-case scenario is us going back to what we're used to right now, which is loading data that is 90s delayed.
The BIGGEST accomplishment of this system is the ability to make it so dynamic, you can literally run an instance of your server from any machine, as in - a fallback server.
You can very simply hand a "repack" of your server to a developer and he will be able to run the same instance of your server on his local machine without ever needing to download or even accessing the database.
This will require me to rewrite almost all the commands to support the replication layer, which as I've said is a HUGE project by itself. For now, the focus is to narrow the startup time to mere milliseconds and see if I'm actually able to restart the server and still maintain the exact same data on a repetitive basis.
As of right now it's all theoretical and has not been implemented in-game. I kindly ask you not to close this thread as I will post here the progress being made on my development machine for documentation purposes.
Feel free to ask questions and share your concerns, I will base my implementation of this system based on this. And no, as of right now it is not in a public repository and will not be at least until I implement it thoroughly, which can take days or even weeks.
Best of luck to me, lol
