=============================================================================================================

Wednesday, December 8, 2010

microSD FAT32 testing using Visual C++


Hi friends,

this post presents a way for testing and learning the FAT32 system on microSD/ SDHC cards without building the hardware with microcontroller, thanks to Henry Yiu.

This project uses the FAT32 library available in my previous post, but does away with the microcontroller part. So, you can use this code with a PC and USB card reader and get insight of the FAT32 data structure by accessing data from the card in raw as well as in FAT format.

This code can be used as a confidence building step for those who are not comfortable with directly building the hardware part. Once the FAT is understood, one can go ahead with development of hardware and customization of the software.

This may look like hardcore software thing, but it's not so. It can be tested by anybody with little knowledge of programming and determination to understand the FAT / sd card formats.

Step by step procedure for testing of the code is explained later.

Here are the Henry's words for the project:
-----------------------------------------------------------------------
When trying to build a microcontroller to access an SD card, I came across this web site which details the hardware and software of how to use an Atmel AVR microcontroller to access an SD card. But I wanted to test out the FAT32 file system code before actually building the hardware. Therefore, I wrote this code to run on Visual C++ to enable me to test the FAT32 code without actually building the hardware.

Tracing the code can reveal many compatibility problems between how the official FAT32 documentation and how Microsoft Windows actually implement the FAT32 system. There are a few interesting things I found for Windows XP. Maybe you can find more using various trace methods and many different brands of SD cards.

The things I found are, for example:

1. After adding or deleting files in Windows XP, the FSinfo next free cluster upper 16-bit is always zero.
2. Adding new files in Windows XP updates the directory structure, but does not set the next entry to empty.
3. Deleting files in Windows XP does not set the FSinfo structure next free cluster to a lower numbered cluster of the deleted file.

Recommended softwares (All the three are available for free):
---------------------------------------------------------------------------------
Microsoft Visual C++ Express

HxD Hex Editor and Disk Editor
The DevCon command-line utility functions

How to use: Only five files:
MAIN.CC, SDCARD.CC, FAT32.CC, FAT32.H, GLOBAL.H.

Use Microsoft Visual C++ Express to build these files with the default built option. Then you can step through the program to learn the boot record structure, the FAT table, and how clusters are linked together. You can also use HxD Hex Editor (Extras -> Open Physical Disk) to study the SD card content. Since data is cached in Windows, you need to unplug and plug the USB card reader for write or delete command to be visible in Windows. The DevCon utility allows you to do so without the need to actually do a hardware plug and unplug. Then you can test the FAT32 system using a batch file.

Enjoy
BR - Henry Yiu

You can download the complete project folder from here:
microSD-FAT32 with Visual C++

Many thanks to Henry for allowing to share his work

Here is the step-by-step procedure of how to use the code:
----------------------------------------------------------------------------------------
(well, this assumes that you have already done some basic background study on FAT32 and microSD cards, may be by reading a few articles on them, since some theoretical knowledge would really help in making sense of the results you get)

(the folder which you download already contains the .exe file, but we'll assume that it is not there, to make the procedure complete)

1. Install and open VC++ express

2. Create a new project (select 'empty project' when asked). Enter the location and project-name. Here we'll select project-name as 'sdcard', and location as desktop

3. Project window will open. Now, go back to desktop and copy the 5 files (MAIN.CC, SDCARD.CC, FAT32.CC, FAT32.H, GLOBAL.H) from the downloaded folder, into the folder created by the VC++

4. Go back to VC++ window and add the header files and source files into respective categories displayed at left side window panel. The panel will now look like this (here main file is open):


5. Now, from the top menu-line, under 'debug', select 'build solution'. This will build the project files. The build window at the bottom of the page is shown in this figure:


6. Now, select 'start debugging' under the same 'debug' menu. This is to make sure that the .exe file is generated. Okay, now you are ready with the .exe file. It will be stored in the 'debug' folder inside the project directory. In our case directory is 'sdcard' on the desktop.

7. It's time to test the .exe file. So, connect the USB card reader, insert a microSD card (make sure the card is already formatted with FAT32, you can also store some files for checking the read feature of the code)

8. Open windows command prompt. In the command prompt, go to the project directory, 'debug' folder. Now, type 'sdcard' or 'sdcard.exe'. This will display the available options. This is described in this figure:

9. Using the command shown in the above figure, you can select 'read', 'write', 'delete' etc. options, as per the command format. After every command executed, it would display 'success!'. If any error, it would display an error message. Some of the file operations are shown here:


10. Once you have performed some operations in last step by creating, deleting files, you need to remove the card and re-insert for checking out the files. (The re-inserting thing is not required if you use the DevCon utility as suggested by Henry). The files which are created will contain random data, as this code is mainly to understand FAT. What is important is that windows should not display that the file is corrupt. This figure shows the results of operations we did in the last step, by opening the card in windows after re-insert:


11. Now, the next step is to see how to use the HxD Hex Editor to understand what we did in the previous steps: Install and open the HxD. From the top menu-line, under 'extras', select 'open disk', select the drive letter of the card and press OK, or select 'physical disks' & then 'removable disk1'. This will open the card memory area, starting with the first sector of the card. Here in the first sector, you can see how the boot sector or MBR (master boot record) of your card looks like.

12. Using the 'find' menu, search for the text string, the name of any file stored in the card. This will take you to the FAT (File Allocation Table) area of the card. (you need to use 'search' option, as sometimes the FAT sector will be located far away and you'll be tired of scrolling down. Alternatively, using the boot-sector data, you can manually calculate the FAT's starting sector number and directly go there by entering the sector number in the box at upper right corner). Here you can see how files are arranged in FAT, by their names, starting cluster numbers etc. Following figure shows the occupied part of FAT and shows location of 'newfile.txt', which we created in step 9.


This is really useful in understanding the effects of creation, deletion of files on FAT table & FSinfo sector, to gain a real insight of the FAT32. And this understanding helps a lot while debugging your code when you make the actual hardware.

It also helps understanding the differences in file handling by the microcontroller FAT32 code and the windows, as you can check the card data in HxD after using either of them. It'll help in debugging by bridging in the troublesome differences.

Here are the additional testing suggestions by Henry:
------------------------------------------------------------------------
In order to start debugging, you will need to enter command lines. You can either change the #if1 to #if0 in main(), or use the Visual C++ command argument input: In Visual C++, select the "Project", then "SDCard Properties", then select "Debugging", and then "Command Argunments", then enter the command. For example:" -w2 e: test1.txt".

To trace the detail of the FAT32 system, you would go to the end of the getBootSectorData() function and select "Run to Cursor". Then you would record these two important sector numbers:

unusedSectors + reservedSectorCount = starting sector of the FAT table
firstDataSector = starting sector of the data area

Now you can open the HxD hex editor to see how these two sectors and how they are being changed by the program once you start stepping through the program. You can press F10 to single step, or F11 to step into a function, or right click on a line and set a breakpoint, then press F5 to run to the breakpoint. You need to press F5 in HxD hex editor to refresh the display in order to see any changes the program made to the SD card.

Once you have master this procedure, you can try doing file transfer between Windows XP and the program, for example, writing a file in Windows XP and reading it back by the program etc. You can also use HxD hex editior to view how Windows XP changes the card when it create or delete files on the card.


Enjoy the testing, It's really a fun!!!

18 comments:

kool kishore said...

i m tryng to make this project but not able to find camera for this

c328 cam is discontinued

please help me reply to kishoreinvits@gmail.com

http://www.riccibitti.com/witnesscam/abstract.htm

CC Dharmani said...

This code actually deals with software testing of FAT32 libraries and functioning of SD cards.

For your application, any VGA color camera should do, which are available quite easily in market.

Luis said...

im getting an error when i try to write to the sdcard: "error opening memory card". I reformatted the sdcard to fat32 and put it in my computers sdcard port, have any idea why i get the error. The card is sdcard not micro.

Henry said...

Hi Luis,
Are you able to read the card? For example, what happen if you type SDCARD -r E: data.txt?
It is most likely that you have a wrong drive letter for the card slot.

Kanna said...

Hi Sir,

i am not able to comple this code. I am trying some bugs rectification and not able to compile this code. so kindly give a correct code for FAT32 Implementation.

Thanking you sir,

Henry said...

Kanna,
What compile error did you get? If you are using the latest (free) Visual C++ Express download from Microsoft, all you need are three C files and two H files to be put into your project, then you just use the default compile option. If you are using other compiler, then you might need to add some runtime support libraries.

moh_ibnumalik said...

Nice website for microcontroller project. Thanks for your effort.

Regards
Mr.Moh Ibnu Malik
http://microrobotics.blogspot.com

Anonymous said...

i am beginner...my project is "implementation of Fat32 file system on sd card by using Atmega8"
plz give me some real time appilications for this project...
give some little bit information about how to implement fat 32 file system on sd card...

umam said...

I want to make program that read sector in SD card. can you help me?

CC Dharmani said...

Hi Umam, function for reading a sector of the SD card is part of this library. Look into the SD_routines.c for 'SD_readSingleBlock()' function. You can call that function to read any sector of the card after the card is initialized. Look into the comments in the code for more info.

umam said...

I mean using VC++. by the way right now I'm porting your sd_routines to cvacr. but I have some problem. when I try read single block, the result is zero character. the porting is not done yet. maybe I miss something. and if i try using mmc, the initialization is always failed. can you help me?. coz my project using different type of card.

Umam said...

My bad. i'm reading the wrong sector. your routines works perfectly in cvavr. Thank you Very much.

Anonymous said...

Thanks for your helpful project,
I added the c++ files and header files to visual c++ 2008 but there is some error.could u plz help me?

ahsan said...

Hi,

I have a problem and I want your opinion.

In my case, the data is coming at the rate of 20 k samples/sec and I am unable to write at this faster speed.

When I try to write the data on a faster speed, most of the times the block of data is written very fast but some time the memory gets busy for a duration of around 600 msec for a block of data to program in the main memory.

I am using single block write command and not multiple block command for data because I do not know that how long the recording can take.

Can you please tell me how much faster one can write on SD card memory 2GB? Or What is the way to write at the fastest speed?

By the way, I never use erase command, as I assume that the erase command is not needed?

/Ahsan

Asif said...

Hi Dharmani,
I have successfully detected SD/SDHC card type using M16 Target processor.
I am facing problem in getting boot sector of memory card. I am using firmware version "SD_GCC_V2.4.1_M32" and have 4GB SDHC card.
Is there a need to change firmware for commands to convert block address into bye address for this firmware version..???
Getting invalid values for (MBR boot signature, 55AA)...
I shall be very thank full to U if please guide me.
If there any other need of modification please inform ...

Thanking You in anticipation

Kind Regards
Asif

Asif said...

Hi CC Dharmani,
I didn't exactly find the idea where to separate the data and FAT buffers to achieve high speed and optimization in the firmware..
CAN U PLEASE HELP ME OUT..

Asif

Dwith said...

i have an error which say

C:\Windows\SysWOW64\ntdll.dll', Cannot find or open the PDB file

can you help me out with it

chandrappadvng dvng said...

Great job well done !!!!!!!!