General suggestions regarding Btrieve File Corruption

When investigating a case of Btrieve file corruption, there are four general causes. More than 90% of all Btrieve file corruption cases are caused be problems in categories III and IV.

I. File corruption is being caused by a problem with the Btrieve engine. If this is the case, the corruption should be reproducible. To isolate Btrieve as the cause, you must provide Developer Support with a test case that demonstrates the problem. This test case can be either your application, or a test application based on the code in your normal application. Btrieve cannot be identified as the cause without a test case. Btrieve logging and the Rollforward utility can also be utilized when trying to produce a test case. Similarly, in the Btrieve 6.x NLM environment, the BRECORD utility can be used.

Always make sure you are running the latest version of Btrieve with any available patches applied. Patches can be downloaded from CompuServe, or you can contact Btrieve Technologies Support and request that the patches be mailed to you on diskette. If Btrieve is found to cause the file corruption, the problem will only be fixed in the current or next version of Btrieve. Development will not fix older versions that are no longer shipping.

II. File corruption is being caused by the application. If the application can make Btrieve calls successfully, but data returned by Btrieve is corrupted, or if the workstation hangs, the cause can usually be found in the application. Common causes include overwritten data buffers, position blocks or memory in general. Utilities such as POSCHECK can help track down some of these errors. Various Btrieve debugging utilities are available for download from Compuserve, or you can contact Btrieve Technologies Support and request that the utilities be mailed to you on diskette. 

3. File corruption is being caused by hardware incompatibility or malfunction. Check all the workstation and server hardware. Hardware-related causes can be difficult to isolate, and sometimes may require you to swap components. If the cause can be proven to stem from one particular workstation, it is usually hardware related. When running a NetWare LAN, always make sure server hardware, workstations, LAN cards and any other components are NetWare-certified.
4. File corruption is resulting from interaction with another software component, including other TSRs, memory managers or device drivers at the workstation, as well as LAN card drivers and other NLMs at the server. Remove all unnecessary components and see if the problem persists. If not, add the components back in one at a time to see which one may be causing the conflict. Also, make sure that each component is the current version and is NetWare-certified for the NetWare version running on the server.

The best approach to take when investigating Btrieve file corruption is "isolate and reproduce." Always make sure you start your tests with "clean" files by rebuilding any files that produced an error during your investigation.

Preventing & Analyzing Btrieve Status 2s

Btrieve status 2s (I/O Error) can be caused by many conditions and can signal problems in any part of your environment. At a fundamental level, status 2 indicates that Btrieve is unable to perform the specified I/O against a designated Btrieve file. I/O errors can be the result of a problem with the Btrieve call parameters, file structure, hardware, system configuration, security rights, memory or any combination of factors.

To diagnose the cause, begin by examining the Btrieve function call that returned the status 2. Is the call returning the status 2 a read operation or a write operation? Do consecutive read and write operations also return status 2s. Did the status 2 suddenly appear on a long-running application? 

Next, consider other possible causes. Is the status 2 returned if you move the file to another machine or a different directory or is it a problem with the file? If the file is on a server, can you access it from another workstation?

Status 2s do not always indicate that a Btrieve file is damaged. This article discusses methods for preventing and analyzing problems with your development environment or application that can lead to status 2s. 

Preventing Status 2s

 If your application receives status 2s, do not immediately assume the worst. Start by examining the following list of do's and don'ts. 

NOTE: Numbers in parenthesis indicate a problem specific to pre-6.x versions of Btrieve, such as (5.)

1. Stay up-to-date on patches for your database products. If the patches do not fix the problem, they may change the symptoms of the problem and provide a clearer understanding of the cause. Patches are available either through the BTRIEVE forum on CompuServe, or by contacting Btrieve Technologies Support. Apply patches first because your problem may already have been fixed.

2. Btrieve files should be flagged NON-SHARABLE to NetWare. Btrieve should be a file's only user. In this configuration, Btrieve controls access to the file from multiple applications and preserves file consistency. The NON-SHARABLE flag prevents different versions of Btrieve from accessing a file simultaneously. For example, you cannot use Btrieve for DOS and NetWare Btrieve (NLM or VAP) on the same file concurrently. The NON-SHARABLE flag also prevents you from making backups of your file while Btrieve has the file open, which could cause an inconsistent Btrieve file being backed up. 

3. Certain hardware and shell problems can trigger Btrieve status messages. If your application suddenly starts receiving unusual status messages:

  * Make sure all hardware is certified by Novell for your version of NetWare. If you have doubts, please contact your NetWare Reseller or Novell Support.

  * Check for hardware failure by running your application on another server or workstation, or from another directory. Experiment with substituting hardware and network cards.

  * Install the newest drivers and shells. Check for patches to the drivers, shells, or network OS.

4. Some viruses can generate status 2s. For example, virus 1704 sends erroneous hardware errors to your application. Check all disks and hard drives for viruses regularly.

5. If your computer has expanded memory and you are experiencing problems, try loading Btrieve for DOS with the "/e" parameter. This parameter keeps Btrieve data buffers out of expanded memory, which may be necessary if another application uses that memory without restoring pointers properly.

6. Make sure position blocks retain physical integrity throughout your application. Check where the position block is defined and what variables are defined immediately before it.

Identify when, where, and how those variables are accessed to make sure the position block is not overwritten.

7. Make sure each Btrieve 5.x format file has only one pre-image file. Multiple pre-image files for a Btrieve file can invalidate file rollback operations.

*If users redirect pre-image files to another directory with the Btrieve "/i" parameter, all users of a Btrieve file must direct their pre-image files to the same directory.

 * With versions of Btrieve prior to v5.00, do not use NetWare search mappings to find Btrieve files. If multiple users with different mappings access the same file, a separate pre-image file may be created in each users current directory.

8. Make sure each pre-image file is associated with only one Btrieve 5.x format file. Btrieve names the pre-image file with the same name as the Btrieve file with a .PRE extension. Two files named EMPLOYEE.DAT and EMPLOYEE.VAC would have the same pre-image filename, EMPLOYEE.PRE.

9. You may have run out of disk space for your Btrieve write operation. Normally, this will return a status 18 (Disk Full), but Btrieve may return a status 2 under some circumstances.

10. Be certain that enough disk space is available for your pre-image file. Status 2s may be returned on a write operation if you do not have enough disk space for the pre-image file. This point is especially important if you use Btrieve transactions. Within a transaction, each write operation adds pages to the pre-image file, and these pages are not deleted until an End Transaction or Abort Transaction operation is performed.

11. You must assign users the necessary NetWare security rights for all accessed files, including transaction and pre-image files. The rights required for normal, accelerated, or exclusive mode access are READ, WRITE, and CREATE.

12. With NetWare v2.10, v3.10 or greater, flagging files "transactional" and using Btrieve for DOS may result in status 2s if multiple workstations access the same files, TTS is active, and Btrieve files are flagged "transactional." TTS locks every page it accesses, including the Btrieve file header page. If another workstation tries to access a file that is currently active with TTS, Btrieve detects a lock on the header page. Since the lock is not a Btrieve lock, status 2 is returned.

13. Do not attempt to directly access the extended portion of a Btrieve file. You can only access the original portion of an extended Btrieve file. For example, if you have a file called STUFF.DAT and you extend it to a file called STUFF2.DAT (using operation 16), and then try to access STUFF2.DAT directly, Btrieve will return a status 2.

14. On NetWare v2.x, if the number of communication buffers currently in use are at or near maximum, BSERVER.VAP may be logged out. Subsequent Btrieve calls will return status 2. To avoid this situation, increase the number of communication buffers. On NetWare v2.2, the maximum is 1000 and the default is 150. To increase the number to 250, type the following in the SYS:SYSTEM directory:

dconfig net$os buffers:250

15. If you apply patches to Btrieve, with DEBUG.COM, give the Btrieve .EXE file another extension (e.g., ren BTRIEVE.EXE BTRIEVE.PAT). After you apply the patches, rename it back to the .EXE extension. If you do not rename Btrieve while using DEBUG.COM on it, that copy of Btrieve will return status 2 on any files it accesses. (Batch files for Btrieve v5.10 and above no longer use DEBUG.COM, so this is not a concern for current versions.)

16. Within a transaction, if you initiate an Abort Transaction with a single-record, no-wait lock (operation code 221), the next time the file is accessed, a status 14 (Pre-Image Open Error) is returned. You will not be able to open the file in any mode after this; the file must be recreated.

17. If you use the "/o" (Override Criti-cal Errors) load parameter when loading Btrieve for DOS, Btrieve translates OS critical errors like "Drive not ready" to Btrieve status codes. Do not use this load parameter unless your application handles OS critical errors correctly.

18. If you are using single-user Btrieve, do not open a file with more than one position block. Doing so will violate integrity checks, and may lead to status 2s. If a file resides on a network drive, only allow one user to access it at a time.

19. When using Microsoft Basic v7.x, use versions of the Btrieve interface (BC7RBTRV.OBJ) dated November 26, 1990 or later.

20. Using BTRIEVE.NLM v5.11, if one station reads a file while another workstation simultaneously opens the file in a transaction, Btrieve returns a status 2 to the reading station. The operation is successful when retried. This situation does not occur with NetWare Btrieve (NLM) v5.15. (See Bullets, June/July 1991.)

21. Older versions of NETx.COM (v3.02 and v3.20) contain known problems. Get the newest version from Novell's Forum or from Novell Support.

22. If you need to run Btrieve files on large (255+ MB) mirrored drives, use NetWare SFT v2.15c or greater.

23. If you use NCOPY.EXE v2.2 or 3.10 to make a copy of a Btrieve file with the characteristics listed below, the duplicate file will be larger by exactly 65K. 

* The Btrieve file was created specifying pre-allocation.

* The last 1K of the file were zeros.

* The file was .5 MB or larger.

  Btrieve v4.10 or greater will return status 2s when accessing the file.

24. An MSDOS v4.01 bug causes status 2s when a Btrieve file's size is with-in 512 bytes of a 32 MB boundary and the file is expanded (See Bullets, October 1991). Microsoft has a patch for this bug that can be applied to MSDOS.SYS or IBMDOS.COM. To obtain the patch, call Microsoft at 900-896-9000 or 206-646-5108.

25. Using NetWare v2.12, NetWare Btrieve VAP v4.11, and Btrieve transactions, status 2s may be returned on an Insert operation. This situation has been corrected in current versions of NetWare Btrieve and NetWare 2.x.

26. Design your applications so that static subprograms do not use stale position blocks to access files. If a "child" process remains in memory after returning control to the "parent" process and then the "parent" closes a file, that file is closed for the "child" as well as the "parent."

27. If you are using a LANtastic network, or any other network that uses SHARE for managing file and record locks, load SHARE.EXE only at the server. (For the exception to this rule, see #28).

28. Load SHARE at your workstation if you are opening a local file with multiple position blocks, or if you are running multiple applications (or multiple instances of the same application) that open the same local file.

29. If you use a CVIS network with Btrieve for DOS v5.00 and above, opening a file with multiple position blocks may return a status 2.

30. Using Btrieve transaction processing with American Megatrends CNT 286 BIOS v1.3 requires v5.10a or greater of Btrieve.

31. Btrieve for DOS v5.10a may return erroneous error messages when you run it on an AT&T Stargroup network. This problem is being investigated.

32. On NetWare v3.11, make sure the following NetWare OS patches are loaded: FATFIX, ASNCRDFX, SPXLISFX, XMDFIX, TTSFIX (if any files are flagged transactional). On NetWare v3.10, apply READFIX.NLM and CACHEFIX.NLM. These patches prevent some causes of status 2s and 52s. Both patches can be downloaded from Novell's NetWire Forum.

33. Changing NetWare directory rights on a directory where a Btrieve file is being accessed causes status 2s.

34. A GetEqual on Key 1, followed by a GetEqual+200 on Key 0, followed by a GetNext+200 on Key 0 may return a status 2 on the GetNext+200 if the GetEqual+200 on Key 0 failed (for example, with a Key Not Found error). The GetNext+200 is an invalid operation since the positioning was not established from the failed GetEqual+200 call. However, a more appropriate status (such as 8 - Invalid Positioning) should be returned rather than status 2.

35. Make sure all Btrieve files located on a NetWare network have valid NetWare owner names assigned to them. Use the NDIR command or FILER utility to verify this.

 Verifying File Integrity

 BUTIL provides an easy way to determine if a file is damaged. Perform a BUTIL -SAVE on all keys in the file. This operation traverses and validates the specified key path and reports any errors. It can write the records from your Btrieve file to an ASCII file in key order. If you don't want to consume disk space, use the command below to report errors to the screen without creating a text file:

BUTIL -SAVE <Btrieve filename> NUL N <key number> [-O<owner>]

To verify data page links, perform a BUTIL -RECOVER operation on the file. BUTIL -RECOVER is not specific to a key path, so you only need to run it once. BUTIL -RECOVER can also write records to an ASCII file. If you do not to need this feature, perform the recovery with the following command:

 BUTIL -RECOVER <Btrieve filename> NUL [-O<owner>]

If these utilities run successfully your Btrieve file is probably intact. If you determine that your status 2 indicates a damaged Btrieve file, replace the file with a backup or rebuild the file using Btrieve or a Btrieve utility. To rebuild the file, use BUTIL -CREATE to create an empty duplicate of the file that may be damaged. Then, use BUTIL -LOAD to insert the records from the BUTIL -RECOVER or - SAVE ASCII file into the duplicate file you just created.

Determining why and when file corruption occured is the next step. Keep a log of affected files, and the times and dates when you detect corruption, and what code modules were running on each workstation. When performed at regular intervals, these operations can help establish when corruption occurs, and provide insight into the cause of the problem. 

BUTIL.EXE (or, with Btrieve for Windows, BTRTOOLS.EXE) is only available with the Btrieve Developer's Kit. (Server-based BUTIL.NLM is a faster version of BUTIL.EXE.) A third party utility, BCLEAN.EXE, can recover files that are severely damaged. Both BUTIL and BCLEAN can be obtained from the Developer Support Group or downloaded from Btrieve's Forum, Library 5, as BTOOLS.EXE. Bclean can only be used with Btrieve v5.x format files.