PROGRAM TO HELP CATALOG AND ORGANIZE RTPS, RESOURCES AND GAMES

Posts

Pages: 1
Hello,

I have been working on saving and cataloging games, RTP's and Resources as I find them online. I have over 4000 games in my collection now and it is getting harder and harder to find duplicates and copies. The solution I am working on is a Program.

The goal of the program is to make a known file list of every game I can find with their unique hash so that I can easily identify if I have already downloaded the game or not. On top of that I would like to include a lot more functionality that includes, but is not limited to:

  • Finding Missing known Files for the game
  • Allow anyone to scan a folder to find games which are not cataloged
  • Scan and detect files in ZIP|RAR|7Z|Selfextracting EXEs for specific file information
  • Detecting the Engine Used for a game
  • Scanning files for text to help determine the Author
  • Determining if it has RTP files included and which version of the RTP is it
  • Determine if you have the same game twice, ignore thumbs.db and save files and only focusing on the main files required for the engine
  • Removing RTP files from within zip without unzipping
  • Rename file to a neatly formatted name if it is known
  • Detect if the game is encrypted or not
  • Anything else that might be useful...

Right now I am working on compiling information on things like this:
(This is not real data per-se, just an example)
{
"engine": "RPG Maker 2000",
"abbr": "RM2K",
"hasRTP": true,
"requiredFilesExts": [
"lmt", "lmb", "lmu"
],
"knownUniqueFiles": [
{
"defaultName": "game.exe",
"hash_sha256": "..."
}
]
}

Additional RTP Info:
{
"engine": "RPG Maker 2000",
"abbr": "RM2K",
"RTPs": [
{
"versions": [
{
"description": "First official RTP released.",
"number": "1.0.0",
"officialRTP": true,
"mediaFiles": [
{
"path": "/battlebacks/grass.png",
"hash_sha256": "..."
}
]
}
]
}
]
}

Now as to WHY I am posting this...
I am looking for anyone knowledgeable that can help list out the following things.

  • Known custom and default RTPs and versions if they are updated and where to download them, so I can install and generate hashes for it.
  • Known EXE's, Patched EXE's, DLL's and Patched DLL's with their version info if available and SHA256 hashes (I plan on scanning these with the games I have, but the more info I have up front, the more I can verify with my data)
  • How to 100% determine between RM2K and RM2K3 without using the exe or folder names, as RM2K3 doesn't NEED the background2 folder for example and some games don't include the exe.
  • Other file information like, RM2K requires lmu, lmt and lmb files to function and every other file allowed like, wav,mp3,ogg etc...

Any information you can give will help me when I complete the programming on my side. Let me know if you want to help, or just post in here with any information you can share.
Documentation
Getting RM2K/3 Games to run without RTP
EasyRTP Documentation

holiness1.wav:

RM2K2/3 Information:
  • Fonts need to be installed for many games, including RMFont for the built in one
  • Detect RM2K3 by the executable hash OR if folders including
  1. Battleback2
  2. todo...when home add the folders
  • Need to figure out how to check if RM2K3 WITHOUT the exe or folders...

RMXP Information
  • Fonts need to be installed for any games utilizing them

RMVX
  • Fonts can be placed in the "Fonts" folder.

RMVXA
  • Fonts can be placed in the "Fonts" folder.


Known RTPs
RM2K RTPs Known
  • Official RTP (v1.00) (English)
  • Official RTP (v1.05e) (English)
  • Official RTP (v1.07e) (English)
  • Official RTP (v1.30) (English)
  • Don Miguel's Custom RTP

RM2K3 RTPs Known:

RMXP RTPs Known:
  • Official RTP (v1.00) (English)

RMVX RTPs Known:
  • Official RTP (v1.02) (English)
  • Official RTP (v1.02e) (English)

RMVXA RTPs Known:
  • Official RTP (v1.00) (English)


Known Runtime Files
RM2K Executables
Default NameVersionCRC32SHA256Entry PointCode Size
RPG_RT.exe1.00...todo...todo0x4972200x96400
RPG_RT.exe1.00 (05/07)...todo...todo0x4972D80x96600
RPG_RT.exe1.00 (06/19)...todo...todo0x4979400x96C00
RPG_RT.exe1.00 (07/11)...todo...todo0x497B500x96E00
RPG_RT.exe1.00 (11/13)...todo...todo0x4975E40x96800
RPG_RT.exe1.00 (11/15)...todo...todo0x4975EC0x96800
RPG_RT.exe1.10...todo...todo0x4977100x96A00
RPG_RT.exe1.50...todo...todo0x49CA800x9BC00
RPG_RT.exe1.51...todo...todo0x49CC300x9BE00
RPG_RT.exe1.60...todo...todo0x49CC780x9C000
RPG_RT.exe1.61...todo...todo0x49D7080x9CA00
RPG_RT.exe1.62...todo...todo0x49D9300x9CC00

RM2K DLLs
  • ...todo

RM2K3 Executables
Default NameVersionCRC32SHA256Entry PointCode Size
RPG_RT.exe1.00.0...todo...todo0x4BE3F40xBD600
RPG_RT.exe1.00.0 (???)...todo...todo0x4BFF500xBF200
RPG_RT.exe1.00.0 (???)...todo...todo0x4BF6A80xBE800
RPG_RT.exe1.02.1...todo...todo0x4C00CC0xBF400
RPG_RT.exe1.03.0 (1)...todo...todo0x4C0B900xBFE00
RPG_RT.exe1.03.0 (2)...todo...todo0x4C18780xC0A00
RPG_RT.exe1.04.0...todo...todo0x4C14D80xC0800
RPG_RT.exe1.05.0 (1)...todo...todo0x4C81C40xC7400
RPG_RT.exe1.05.0 (2)...todo...todo0x4C81F00xC7400
RPG_RT.exe1.06.0...todo...todo0x4C98040xC8A00
RPG_RT.exe1.07.0...todo...todo0x4C99840xC8C00
RPG_RT.exe1.08.0...todo...todo0x4C9C1C0xC8E00
RPG_RT.exe1.09.1...todo...todo0x4C9D240xC9000

RM2K3 DLLs
  • ...todo

RMXP Executables
  • ...todo
RMXP DLLs
  • ...todo

RMVX Executables
  • ...todo
RMVX DLLs
  • ...todo

RMVXA Executables
  • ...todo
RMVXA DLLs
  • ...todo


Known Patches
Reference For later...
RM2K
  • ...todo

RM2K3
  • ...todo

RMXP
  • ...todo

RMVX
  • ...todo

RMVXA
  • ...todo
Hi krum110487,

I just wanna chime in here quickly. You've done an incredibly amount of work in the past and I eagerly look forward to where you take this ❤️.

most of this info you probably already know. so I apologize if I'm repeating myself. but over on EasyRTP, they have a RTP reference library for 2000 and 2003. Going over the various patches/resources don miguel added to 2000 and the official RTP, make sure to look at both entries 2000 and 2003 as there's a lot of info covered.

There's also some info not covered like this
https://rpgmaker.net/forums/topics/22883/?post=946422#post946422

author=LordBlueRouge
I found Royal.png from Lord of the Rings (Author)(RM2k)(2005)(wRTP).rar

My memory of RM2000 is incredibly foggy, as I only started playing around with RM2000 around 2004.
I was too busy with school to really get into RM at that point

But I feel like, similar to SegNin solving the mystery of holiness1.wav there was a subsection of RM2000 files that everyone had used during the early 2000s, that they used alongside Don Mig's RTP?

EDIT: Yeah, I just, checked the link Darken provided. Royal.png is not part of rm2k_rtp, that's part of Don Mig's RM2000 installer.
The RM2000 RTP we have under the masterlist is actually from RMTutorials/RMRK/2DRPG.com, which doesn't have these files. I still have Don Mig's RM2000 installer (from several harddrives ago and on CDs). I'll see if I can put together a link or something that fixes these issues.

EDIT2: It'd be cool if we could create an injector that'd just, install those additional 38 files from Don Miguel's installer sans font and call it
"RTPe2: the don miguel files".


But ya, my info on RTP stuff is severely limited. don mig is on twitter. and his website is still around. he might be able to fill in a few blanks. I also remember coming across other RTPs for French and Spanish I think? But I'd need to go through old harddrive in order to recall.

Thank you doing this. definitely looking forward to your progress on this.❤️

Excellent work as always!
saved for compiling data from responses.
author=LordBlueRouge
Hi krum110487,

I just wanna chime in here quickly. You've done an incredibly amount of work in the past and I eagerly look forward to where you take this ❤️.

most of this info you probably already know. so I apologize if I'm repeating myself. but over on EasyRTP, they have a RTP reference library for 2000 and 2003. Going over the various patches/resources don miguel added to 2000 and the official RTP, make sure to look at both entries 2000 and 2003 as there's a lot of info covered.

There's also some info not covered like this
https://rpgmaker.net/forums/topics/22883/?post=946422#post946422

author=LordBlueRouge
I found Royal.png from Lord of the Rings (Author)(RM2k)(2005)(wRTP).rar

My memory of RM2000 is incredibly foggy, as I only started playing around with RM2000 around 2004.
I was too busy with school to really get into RM at that point

But I feel like, similar to SegNin solving the mystery of holiness1.wav there was a subsection of RM2000 files that everyone had used during the early 2000s, that they used alongside Don Mig's RTP?

EDIT: Yeah, I just, checked the link Darken provided. Royal.png is not part of rm2k_rtp, that's part of Don Mig's RM2000 installer.
The RM2000 RTP we have under the masterlist is actually from RMTutorials/RMRK/2DRPG.com, which doesn't have these files. I still have Don Mig's RM2000 installer (from several harddrives ago and on CDs). I'll see if I can put together a link or something that fixes these issues.

EDIT2: It'd be cool if we could create an injector that'd just, install those additional 38 files from Don Miguel's installer sans font and call it
"RTPe2: the don miguel files".


But ya, my info on RTP stuff is severely limited. don mig is on twitter. and his website is still around. he might be able to fill in a few blanks. I also remember coming across other RTPs for French and Spanish I think? But I'd need to go through old harddrive in order to recall.

Thank you doing this. definitely looking forward to your progress on this.❤️

Excellent work as always!


Thanks for this, what would help most would be links to any known patches for exe's, known RTPs with a good description of it, etc.

That way I can download, install and catalog the files so I can hash all of them to ensure I have a clean set. Then I can compare hashes to figure out which file are unique to whatever sets. The more unique installers and sets I get with information about them, the more accurate I can build a dataset! I am happy to do the legwork of installing and hashing the files, it is just a matter of not knowing about specific custom RTP's or some niche version of them.

Once I have that, I think I will have the program allow you to switch to RTP's on the fly via registry changes. I kinda already do this for my own pc, but they are just .reg files that update it to a specific folder where I have the files.

Okay.

This is what I have from the archive thread. The first two are from Carius's RMTutorial's from 2007-ish? Be very careful when installing the RTP for 2000, because if you already have RTP installed, It sometimes nukes your RMfont file, making it blank or unreadable This is a problem I wanted to sort out but haven't had a chance to fix yet. The easiest way to fix it is to use a restore point at the moment.

the mirror's are from a locker RMN.

author=LordBlueRouge
-OLD rpgmaker2000/2003- RTP (required for some games)


Don Miguel's RM2000 RTP(mirror) - - - - - - RPGAdvocate's RM2003 RTP(mirror)
Note:How to make your rpgmaker games playable without RTP
EasyRPG RTP Reference List
2022 Update: Some older games use holiness.wave (more info)
author=LordBlueRouge
Okay.

This is what I have from the archive thread. The first two are from Carius's RMTutorial's from 2007-ish? Be very careful when installing the RTP for 2000, because if you already have RTP installed, It sometimes nukes your RMfont file, making it blank or unreadable This is a problem I wanted to sort out but haven't had a chance to fix yet. The easiest way to fix it is to use a restore point at the moment.

the mirror's are from a locker RMN.

author=LordBlueRouge
-OLD rpgmaker2000/2003- RTP (required for some games)


Don Miguel's RM2000 RTP(mirror) - - - - - - RPGAdvocate's RM2003 RTP(mirror)
Note:How to make your rpgmaker games playable without RTP
EasyRPG RTP Reference List
2022 Update: Some older games use holiness.wave (more info)

Thanks for these, I install using sandboxie, so I am not worried about overwrites! Although I should include the fonts in the RTP, now that you mention it.

Now for any other more unknown RTP's that exist and for EXE versions and EXE patches and DLL's and Patched DLL's that is where I am much more fuzzy, RTP's I have a decent start / handle on!

Thanks again LordBlueRouge!

ps. Is it possible to make a nested list on this forum? (nvm, I got it)
author=TheRpgmakerAddict
Dunno if can help but I have these links from an old italian site, the Phantom Castle.

http://www.phantomcastle.it/phantom/rpgmaker/rpgm2k/rtp.exe
http://www.phantomcastle.it/phantom/rpgmaker/rpgm2k3/CommonRTP.exe


Thank you! if anyone can DM me old RTP's and Old RPGMaker installers via DM I would like to collect as many versions as I can so I can confirm all the different files based upon the official installers. I have updated the known executable values so I can identify a version of the executable found above.

So I should be able to determine between RM2K and RM2K3 using the entry point as well as other fields.
Orochii linked the Spanish RTP on discord.
https://rpgmaker.es/descargar/

I know there's a french one and a german one as well. when I have some more free time I'll start digging around in an old harddrive.

EDIT: Now I remember. the English Translation for the rpgmaker game "Taut", originally a German rpgmaker game, needed 2000 RTP. However, the french translation needed 2003 RTP. I never got around to solving this or fixing this. But I made a note of it, because this probably what something like this would help solve.

Same game, two different languages. two different RTPs. I'll see if I can find a link if needed.
I have been working on this, I think I have a design that will allow me to use this for multiple platforms and roms, much like the other romset managers out there. But those don't really handle multi-file setups very well and I hate how unusable they are.

I think I have a good simple design, not pretty, but dead simple. I am working on a streamline process to ensure the entire archival process is clean and fast. I am in the early stages still, but I just wanted to update that I am still working on this.

Right now, it will have an "Archive Group", this will be like "Consoles", "Personal Computers" or "Creation Engines/RPG Maker" (creation engines are engines that share an executable between games)

Then under the groups will be "Engine" which is completely optional, but in the "Creation Engine's" case, we set it up as "RPG Maker"

Then under the engine will be "Platforms", so "SNES", "PC", "Windows 9x" or "PS1" etc.... (This is next because there are Creation Engines on PS1 and other systems, and this allows for the roms / PC to specify the OS)

Maybe an optional in-between here named "Media Type" which would be "Games", "Applications", "Books"... then it would have the set name like no-intro vs tosec vs redump, so "Games" could hold a combination of the 3 sets below it... not sure yet...This could possibly be AFTER the sets...I am still debating this.

Then there will be "Sets", this will be like "No-Intro" for the rom scene, or "Games" / "Resources" / "RTPs" / "Editors" for PC stuff, each of these are customizable to the set needs as well as the ones above. I may allow nested sets, but I haven't decided if I need these yet.

All of this combine could look like...
  • Personal Computers/DOS/Games
  • Personal Computers/DOS/Applications
  • Personal Computers/DOS/Books
  • Personal Computers/Windows 9x/Games
  • Personal Computers/Windows 9x/Applications
  • Personal Computers/Windows 9x/Books
  • Creation Engines/RPG Maker/PC/Games
  • Creation Engines/RPG Maker/PC/RTPs
  • Creation Engines/RPG Maker/PC/Resources
  • Creation Engines/RenPy/PC/Games
  • ROMs and ISOs/SNES/Games
  • ROMs and ISOs/Sega CD/Games
  • ROMs and ISOs/Commodore 64/Games
  • ROMs and ISOs/Commodore 64/Applications

Each tier will have a configuration on things such as "tools" to do thinks like convert xyz files to png for review, anything specific to that particular "Group", "Engine", "Platform" or "Set". This will allow for this to be used with other PC creation tools like renPy for example.

Along with the Tools configuration, I want baked in recognition tools to determine the version of the tools used, for example renPy v1.2.3 was used to built X game, as well as any other identifiable information to be used in archiving.

The steps to archival goes like this.

  • 1) Download a file to the "Download" folder specified.
  • 2) If the Program is open, it will recognize a new file and try to determine the type by opening the file (anything that 7z can handle, this program can look into, including MSI installers and ISOs)
  • 3) If the file is identified as an "RPG Maker" game for example, it will move it from the download to rpgmaker folder specific "./RPG Maker/PC/Games/Needs Review/Detected" folder, if it is NOT automatically detected based upon the settings, it will be moved to "./Downloads/Manual Review"
  • 4) At this point you will use the tool to review the information inside of the compressed file / installer. I would like to use the MSI command line to extract the files in a controlled environment without registry doing its thing. So I will probably use that to get the files extracted that way, then immediately zip the contents...but I am need to investigate this part, installers will be a pain.
  • 5) Once you have the information filled in for all the meta data and you are confident, you will move it to "Gold Data" folder, via the tool. "Gold Data" will hold all of the "Curated" games/files you are archiving. During the entire process of starting at the "Download" folder it will check to see if you have this game already and keep a hash of the original zip so that if you happen upon the same exact zip again with a new name, it will be immediately recognized as a duplicate.
  • 6) All items in gold data have been detected or manually reviewed and files compared to ensure there are no duplicates.

I want to include the option to scan a hard drive for games and move them to the downloads folder auto-magically, for people to use to help find new content, for example Segnin wouldn't have to do manual checks, we could pull everything of his we still need in a few clicks instead of manual labor they have to do now.

I also want the ability to import existing DATs used in all those other ROM managers. As well as the ability to merge DATs so that 1 DAT can generate any set type you want and each file can be determined which set they are part of, where their is a main dat that is used as the main one...not 100% sure on this yet, still need to review.

Let me know if you have any ideas or anything else I should consider in the process or anything else!
Pages: 1