I'll start with what you absolutely need to call yourself a real otaku, and work my way down to more rarefied ingredients.
Soy sauce (shoyu) - I'll slap you if I have to tell you what this is. Soy sauce replaces salt in Asian recipes. You can buy it in dark, light, reduced-salt (what's the point?), and flavored varieties, but for most recipes, the plain stuff is just dandy. Buy it anywhere; even Winn-Dixies in East Jesus carry Kikkoman soy sauce. Any brand is good.
Rice vinegar - Sweeter than wine vinegar, which is what Westerners generally use. Buy it anywhere; chances are good that any large supermarket will have twelve-ounce bottles of Marukan rice vinegar shelved with the other vinegars, or with the Asian foods, or both. Any brand is good.
Mirin - Also called aji-mirin. A sweet, syrupy wine made from sake, used the way honey is used in Western cooking. (Mirin is runnier than honey and has its own distinctive taste, though, so honey is a poor substitute.)
Dashi - Soup stock made from bonito flakes. Dashi is the foundation for hundreds of Japanese soups and sauces, including miso. You can be authentic and make your own from bonito flakes, or you can buy powdered dashi concentrate. Dashi comes in primary and secondary forms, depending on how often the bonito flakes have been used; if you're using powdered stock, read the back of the package to find out how much powder to use to make the right type of stock for the recipe. Buy it at an Asian store, where it will probably be shelved somewhere near the rice and soup flavorings. Occasionally supermarkets carry powdered dashi. I don't know how the brands stack up, but I use Shimaya Bonito Flavored Soup Stock and have no complaints.
If you have soy sauce, rice vinegar, and mirin, you can whip up an astounding number of dipping sauces--even more if you have dashi.
Japanese rice (kome if it's uncooked, gohan if it's cooked) - Very short-grained, very sticky white or brown rice. (Most Japanese eat white rice; brown rice is considered health food.) The VERY short-grained, inhumanly sticky sweet Asian rice called "mochi" is sometimes sold as Japanese rice, but it's too sticky to eat for dinner; Asians pound it into a paste and make sweets with it. Buy it at a large supermarket, where it will probably be called "sushi rice" and be sold in small bags, or at an Asian store, where it will lie in huge piles of 10- and 20-pound bags and be much cheaper. All brands are good, but a discriminating palate can detect slight differences. Experiment until you find a brand you like.
Miso - The fermented soybean paste which gives miso soup its flavor, and which pops up in Japanese cooking in everything from roasts to salads to dips. Miso shades from chocolate-brown to pale yellow; the darker the miso, the higher the sodium content. (And miso is very salty.) Lighter misos are called shiro-miso, or white miso; darker types are aka-miso, red miso. Like beer, the exact taste of a type of miso depends on the bacteria involved in the fermentation, so each region has its own specialty miso. Unfortunately, most of these misos don't make it across the pond. The average Asian grocery has maybe two types of miso, three if you're lucky. Fortunately, you don't need anything fancy. If you're just starting out in Asian cooking, buy the smallest container of a middling-red--orange, really--miso; you'll probably end up with about a pound of miso for about $4. It looks small, but it will last you for over a year.
Nori - Thin sheets of nori seaweed used to wrap sushi, onigiri, crackers, and many other things. Nori comes both toasted and untoasted; if you buy the untoasted and the recipe calls for toasted, turn on the stove and hold the sheet of nori over the flame for a second. (It takes almost no time to toast.) Buy it just about anywhere; the trendiness of sushi means that any supermarket with even the poorest excuse for an Asian section will stock at least one brand of nori.
Kombu/Konbu - A kind of kelp which is used to flavor broths and sauces. It starts off looking like a grungy dried banana leaf, and turns into a massive, pulsating tentacle which hangs out both sides of the pot, writhing and thinking dirty thoughts about your Rei figurines. Japanese porn is instantly comprehensible when you use kombu for the first time and see what every Japanese child grows up seeing in Kaachan's kitchen. If you're still willing to use kombu after that introduction, wipe both sides of the dry kombu gently with a damp cloth, taking care not to wipe all of the white powder off--it's the powder which provides much of the flavor. (The kombu is safe to go near at this point, since it has no power in its dry state.) Place the kombu in the cooking water or broth to soak, and witness its mighty expansion. Two minutes before the recipe tells you to remove the kombu from the water, pick up your weapons and psyche yourself for battle. You will need both minutes. At the appointed time, enter the melee. When you have wrestled the kombu from the water and it hangs, limp and flopping, from your knife, nail it above the kitchen lintel and mock it while you bandage your wounds. Buy it at a Japanese grocery. Groceries of other nationalities are sometimes foolhardy enough to stock this menace to life and virtue, and American supermarkets occasionally indulge in fatal innocence concerning this ravage, but you'll have the best luck at a Japanese store. Kombu is usually near the nori and wakame, where it terrifies the delicate and sensitive wakame into fits.
And that's it. The rest of these ingredients are common but not essential.
Daikon - Also called "Japanese radish." It's generally sold in chunks as big around as your arm, and if you get the whole daikon, you can brain kombu with it. Daikon is essential to Japanese cooking and is eaten both raw and cooked, but I have no expereince with it. This bit gets filled in later. Hey, I warned you that the site wasn't done, didn't I?
Fish sauce - A strongly-flavored brown liquid used heavily in Southeast Asian cooking. Japanese food doesn't call for it much, but it's worthwhile to try out. Put a few drops in your ramen.
Gyoza - Japanese pot stickers, smaller and thinner-skinned than the Chinese kind. They're traditionally filled with pork and vegetables, but you can get them in any flavor, including kimchee and vegetarian. I buy Day-Lee Pride brand gyoza, which come 48-50 to a $7 bag. Other kinds of dumplings can be substituted for gyoza--there are dozens of kinds of Korean dumplings on the market.
Inari-no-moto - Little pita pouches of fried tofu canned in mirin syrup or soy sauce. Essential for making Inari-zushi.
MSG (aji-no-moto) - Japanese food is full of MSG. A lot of foods, like powdered dashi, come preseasoned with MSG; other recipes call for a spoonful or two. In Japanese, MSG is called "aji no moto," which means "the origin of taste." (I think Americans would be far less afraid of MSG if it had a pretty name like "aji no moto," instead of a clinical, chemical name like "MSG.") And it is yummy. Add it if you can. MSG is a grainy white powder which is sold in bags and shaker cans. Buy it at an Asian store, or risk censure and ask for it at a large supermarket.
Oden - Japanese fried fish-paste cakes; they come in shapes ranging from flat sheets to round tubes to little nuggets, and are usually varying shades of tan. Very yummy once you get past the Western taboo against fried fish paste. Don't confuse oden with udon.
Pickles (tsukemono) - Not American pickles, Japanese pickles. The Japanese pickle everything which doesn't fall apart in vinegar, which makes for an amazing variety of pickles. Pickles are central to the Japanese diet, to the point that monks traditionally eat little but rice and pickles. Unfortunately, Japanese pickles haven't made it across the pond. Why this should be, I don't know, but perhaps it's because a lifetime of nothing but salty waffle-cut cucumbers spread limply over hamburgers doesn't prepare the mind for a true pickle experience.
(Now you understand what Bishonen Boy was screaming about in the Ranma movie.)
The Japanese word for "pickle" is "tsukemono." Pronounce it carefully. I learned "tsukemono" from the Ranma movie back when my command of Japanese was, to put it charitably, not so good. A Japanese co-worker asked me whether I knew any Japanese, and I said, "Just a few words--baka, hai, oni, tsukemono." She went beet red and asked me where I learned such words. "Children's shows!" "That last word, what does it mean?" "Er... pickles?" "OH! Tsukemono!" "Wasn't that what I said?" Beet red again. "Well..." It took me several minutes to find out that I'd actually said "sukimono," "nymphomaniac."
Ramen - Don't make me slap you.
Raw fish (sashimi) - DO NOT BUY JUST ANY RAW FISH AND SLAP IT ON RICE! You'll make yourself sick! Sushi-grade raw fish is very fresh and has been inspected to ensure that it is parasite-free. Some kinds of fish, like catfish, are never parasite-free, so they are never eaten raw. If you want to make sashimi, buy sushi-grade fish from a Japanese grocery or from an American supermarket which knows what it's doing, and eat the fish right away. If you can't get sushi-grade fish where you live, substitute smoked fish. It's not perfect, but it's close enough, and it's better than a parasitic infection.
Seafood balls - Balls of cooked seafood paste about an inch across. They come in fish, cuttlefish, and shrimp varieties. Delicious, even eaten cold straight from the package. Buy Venus brand; the other brands often have bits of bone or shell in them. (These might be Southeast Asian rather than Japanese. The Japanese version, oden, isn't quite the same.)
Sesame oil - A strongly-flavored oil used as a seasoning. Don't use it for frying! Everything you put in the pan will taste like sesame.
Somen - Fine noodles made of wheat flour. Somen is a summer dish, and is usually eaten cold.
Tofu - Blocks of white soybean goosh, and the reason that a land-hungry society managed to stay well-nourished despite an almost total lack of beef, pork, and chicken. Tofu soaks up the flavors of the food around it. Tofu comes in several consistencies, from hard tofus which can be fried to soft and silky tofus which are best blended into a paste and used in sauces.
Udon - Thick noodles made of buckwheat. Udon is sold dried in packages, or moist and tightly-sealed as a ramen-like instant meal. Don't confuse udon with oden.
Wakame - A thin, frilled kind of seaweed which is used in soups and salads. It's hard to find outside of Japanese groceries, where it's stocked near the nori and kombu. Buy the bag nearest the kombu, since that bag will have the curliest wakame.
Wasabi - Because every otaku worth his or her salt eats massive gloppy mounds of wasabi on their sushi. If you've never eaten wasabi before, don't be put off by the rumors; it's actually a sort of sweet paste. That's why it's that bright, candylike green. Wasabi is sold both premixed and powdered. It's also sold fresh, but fresh wasabi is hard to find in the States.