Hmmm... I can certainly throw in a quick and dirty tutorial. You have to know the chipset you're using first of all. Typically with the refmap ones, the tree canopy itself has 12 edge tiles (4 being your inside corner tiles), a center upper level tile, and a center lower level tile. The lower level tile is actually key for layering multiple edges. You want to make sure that one has no passability while the upper level version has overhead passability. There's a couple different trunk styles you can use, but they all kind of work the same way and should have no passability.
I start out with the center canopy tile (usually just a solid dark color) and make the full forest outline, factoring where I want the player to go, and what they'll have access to do. It doesn't have to be perfect, just make sure there's enough room to place trunk tiles and other decorative tiles. Decide if you want things like hills and cliffs at this stage too, because those will get harder to add in later. From there, go through canopy edges in the upper layer, adding secondary edges if you please. Then add in the rest of your details, thinking about what's going to overlap what.
From there, you can focus on the ground, using 3-4 different styles from dirt, to thick grass, to different shades of grass. I basically just place them at random, with dirt sort of focusing on where you want the player to go (since it's recognized as the path most traveled).
I usually skip things like adding birds and butterflies and crap because you don't want it looking too busy. Just busy enough to be interesting. If you insist on adding birds and other creatures, I would place them off to the side of the paths unless you want to use them as enemy encounters.
Also, I would say don't make your forest one huge giant map because players will probably get bored real quick trying to navigate that. I actually prefer using smaller maps and separating the forest by a couple different paths, giving each a purpose. A lot of this process is using good judgement and thinking about what the player will be thinking playing it. It takes a lot of practice to get good though. Good luck!
Yes, they're custom. I usually use the REFMAP/First Seed Material tilesets as a base (with some exceptions), and then use Photoshop to edit/combine the heck out of them. Original graphics are added where appropriate. I occasionally use rips, but mainly for cool looking objects that I can't find anywhere else, and then I give them my own flair.