====== What is SMART ? ====== SMART/S.M.A.R.T stands for Self-Monitoring, Analysis and Reporting Technology. It is basically a system that collects information about a hard disk drive (HDD) and solid state drive (SDD), and allows you to run some tests on the drive to determine its approximate health. It is important to note that SMART is far from perfect. Although a failed "Pre-fail" SMART attribute predicts failure, having no failed attributes does NOT mean the drive is not failing. The drive can be failing with above threshold attributes. This leads us to the next section backing up your data. ====== Backing up your data ====== According to [[https://www.us-cert.gov/security-publications/data-backup-options|CERT]] you should follow the 3-2-1 rule: 3 - Keep 3 copies of any important file: 1 primary and 2 backups. 2 - Keep files on 2 different media types to protect against different types of hazards. 1 - Store 1 copy offsite (e.g. outside your or business facility). In summary, keep 3 backups: 1 primary, 1 onsite, 1 offsite. This is of critical importance because your device can fail at any time without warning and for various reasons. Backing up your data is the only way to be reasonably sure that you won't lose it. You CANNOT rely on SMART to reliably tell you when your HDD is going to fail and to do so in due time to allow you to save your data. ====== SMART Attributes ====== In order to be able to use SMART you need: - A HDD or SSD that supports SMART - SMART enabled in the UEFI/BIOS - Software to interface with SMART Some commonly used software to interface with SMART is [[http://www.smartmontools.org/wiki|smartmontools]], or you can find individual manufacturer's utilities on [[https://www.ultimatebootcd.com/|UBCD]]. Some people prefer smartmontools because it is easily accessible from the command line. Others prefer the manufacturer's utilities because they sometimes have more features than smartmontools. Which is better is mostly down to user preference and the details of the situation. For this article we will focus on smartmontools and more specifically smartctl. In order to display the SMART attributes with smartmontools you need to run the following as root: smartctl -a /dev/sda Note that we will be assuming that ''/dev/sda'' is your HDD/SSD device node. In many cases this is the first HDD/SSD on the system, but you need to double check to make sure it is the HDD/SSD you are interested in. The output will be something like: bash-4.2# smartctl -a /dev/sda smartctl 5.43 2012-06-30 r3573 [x86_64-linux-3.10.63] (local build) Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Model Family: Seagate Barracuda (SATA 3Gb/s, 4K Sectors) Device Model: ST1000DM003-1CH162 Serial Number: Z1D6DR9C LU WWN Device Id: 5 000c50 064a62447 Firmware Version: CC49 User Capacity: 1,000,204,886,016 bytes [1.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Device is: In smartctl database [for details use: -P show] ATA Version is: 8 ATA Standard is: ACS-2 (unknown minor revision code: 0x001f) Local Time is: Sun Jan 4 16:02:08 2015 CST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 584) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 111) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x3085) SCT Status supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 118 099 006 Pre-fail Always - 168101376 3 Spin_Up_Time 0x0003 097 097 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 425 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 069 060 030 Pre-fail Always - 9675211 9 Power_On_Hours 0x0032 096 096 000 Old_age Always - 3982 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 433 183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0 189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0022 071 063 045 Old_age Always - 29 (Min/Max 20/29) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 25 193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 504 194 Temperature_Celsius 0x0022 029 040 000 Old_age Always - 29 (0 18 0 0 0) 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 12154757451688 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 14098900823 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 800819281 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed without error 00% 3005 - # 2 Extended offline Completed without error 00% 2008 - # 3 Extended offline Completed without error 00% 1014 - # 4 Extended offline Completed without error 00% 13 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. This is just an example from my current HDD. Technically ''smartctl -a'' lists everything, not just attributes, but the whole output is more useful than just the attributes. Some things to note on the output is that SMART support is available and enabled. If it is not available then your device may not support SMART, which can occur if this is an external HDD with a cheap enclosure or if the device is not a HDD/SSD. If it is not enabled, go into your UEFI/BIOS settings and enable it. Also note ''SMART overall-health self-assessment test result: PASSED'', it should be PASSED unless your HDD is failing. Note the line ''Auto Offline Data Collection: Enabled'', this is a feature that is enabled by default on modern internal HDDs. ''man smartctl'' explains what this feature does and how to enable it: -o VALUE, --offlineauto=VALUE [ATA only] Enables or disables SMART automatic offline test, which scans the drive every four hours for disk defects. This command can be given during normal system operation. The valid arguments to this option are on and off. This also updates attributes that are marked ''Offline''. Unlike ''Always'' updated attributes, ''Offline'' attributes are only updated if this is enabled or if you run a SMART test. Note also that the approximate times for running various tests are listed. We will discuss SMART tests in the next section. Now about the attributes, their meaning is summarized in ''man smartctl'': Each Attribute also has a Threshold value (whose range is 0 to 255) which is printed under the heading "THRESH". If the Nor- malized value is less than or equal to the Threshold value, then the Attribute is said to have failed. If the Attribute is a pre-failure Attribute, then disk failure is imminent. The Attribute table printed out by smartctl also shows the "TYPE" of the Attribute. Attributes are one of two possible types: Pre-failure or Old age. Pre-failure Attributes are ones which, if less than or equal to their threshold values, indicate pending disk failure. Old age, or usage Attributes, are ones which indicate end-of-product life from old-age or normal aging and wearout, if the Attribute value is less than or equal to the threshold. Please note: the fact that an Attribute is of type 'Pre-fail' does not mean that your disk is about to fail! It only has this meaning if the Attribute´s current Normalized value is less than or equal to the threshold value. If the Attribute´s current Normalized value is less than or equal to the threshold value, then the "WHEN_FAILED" column will display "FAILING_NOW". If not, but the worst recorded value is less than or equal to the threshold value, then this column will display "In_the_past". If the "WHEN_FAILED" column has no entry (indicated by a dash: ´-´) then this Attribute is OK now (not failing) and has also never failed in the past. Thus, the most important attributes are marked ''Pre-fail''. If the value of a ''Pre-fail'' attribute is below threshold, the attribute is failing implying that the HDD is failing. A failing attribute will be marked as ''FAILING_NOW'' or ''In_the_past'' if it has failed now or in the past, respectively. ''Old_age'' attribute failures do NOT necessarily mean imminent failure, but rather that the drive is getting old and it should be monitored more carefully or replaced at some point. For the exact meaning of each attribute, please see the [[https://en.wikipedia.org/wiki/S.M.A.R.T.|Wiki]] page. Some specific attributes that I would like to discuss are as follows: ===== #4 Start_Stop_Count and #12 Power_Cycle_Count and #193 Load_Cycle_Count ===== This attribute is important for laptop HDDs, because they default to powering off when not in use. Now, although laptop HDDs are designed to spin up and down more times than desktop HDD and this is an ''Old_age'' attribute, it still wears down the drive. Unless you run on batteries all the time you may want to consider turning off this feature by adding this to a boot script such as ''/etc/rc.d/rc.local'': hdparm -B 254 /dev/sda ===== #9 Power_On_Hours ===== This is the age of the drive in hours. This is rather important because it tells you how old the drive is and thus how likely it is to fail. HDD failure among other things follows the [[https://en.wikipedia.org/wiki/Bathtub_curve|Bathtub curve]]. As such, the highest failure rate is among very young (infant mortality) and very old (worn out) drives. This is important because I hear many people saying, "Oh, but the drive is brand new, it can't be failing." Wrong, a new drive is more likely to fail than a middle-aged drive, much like an old drive. ===== #174 Unexpected power loss count and #192 Power-Off_Retract_Count ===== Sudden power loss is detrimental to both HDDs and [[http://hardware.slashdot.org/story/13/03/01/224257/how-power-failures-corrupt-flash-ssd-data|SSDs]]. UPS power backups should be used for systems that are on all time for this reason as well as many others. Make sure to also shutdown your computer properly whenever possible to prevent damage and data loss. ===== #190 Airflow_Temperature_Cel and 194 Temperature_Celsius ===== Although many people believe that HDDs should be kept cool and are sensitive to heat, a [[http://research.google.com/archive/disk_failures.pdf|large Google internal study]] suggests that high temperatures are only significantly detrimental to old HDDs. ===== Bad Blocks (#5, 196, 197, 198) ===== Bad blocks are basically areas of the disk surface that are damaged and can no longer hold data reliably. Internally the HDD/SSD deals with these by marking them and remapping/reallocating them to other areas. Bad blocks increase with the age of the drive. It can be expected that you will encounter bad blocks with every HDD and SSD. The question is when does this become something to be concerned about ? That is hard to say, and in general you will have to deal with each device on an individual basis. A large increase in the number of bad blocks could mean the drive in nearing its end. Keep monitoring the ''Pre-fail'' attributes and decide when to change it out. ====== SMART Tests ====== There are 3 main types of SMART tests that you can perform. * short: a superficial test that tests electrical and mechanical performance and updates offline attributes * conveyance: identifies damage during transport (mostly useful for external or laptop HDDs) * long: a short test plus it scans the disk surface for bad blocks These tests are run with the ''-t'' option like: smartctl -t long /dev/sda These tests can all be run on a running system without major side-effects. If you expect the long test to finish, you should minimize HDD usage as it has to scan the whole disk to finish the test. After waiting for the test to finish, you can get the results using the ''-a'' option as shown in the previous section. Short and Conveyance tests should always pass. If these fail, check the attributes as the drive is probably failing. A long test can fail if there are bad blocks, and this does NOT mean the drive is failing. The long test stops when it finds an error on the disk, so if there is a bad block it just stops. This doesn't mean the drive is failing, but you will have to wait for the HDD to remap/reallocate the block, or technically you could try to force it to do so: http://www.smartmontools.org/browser/trunk/www/badblockhowto.xml However, this method is difficult to implement safely, so you should usually just wait for the HDD to remap/reallocate. How often should you run these tests ? That depends. If you run a server then more often is better, the smartmontools site recommends weekly tests. For a home user, I usually run a long test every 1000 power on hours, but that is up to you and also depends on the details of the drive and situation. ====== Is my drive failing ? ====== A failing drive is defined as: - Having a ''Pre-fail'' attribute below or near threshold, marked ''FAILING_NOW'' or ''In_the_past''. - Having an ''Old_age'' attribute below or near threshold, marked ''FAILING_NOW'' or ''In_the_past'' **PLUS** other signs of failure such as consistent failure of SMART tests, strange noises, slowing down, corrupt data, etc. A failed long test does NOT mean your drive is failing, it could be just bad blocks. See previous section. Do not ignore your senses, if the HDD sounds unusual or makes strange noises, monitor it closely and/or replace it. Again, SMART cannot tell you with great accuracy if or when the drive will fail. The drive can fail with above threshold attributes and minimal signs. The only hope you have to keep your data safe is to backup your data, use the 3-2-1 strategy as mentioned above. ====== smartd ====== What is smartd ? It is a daemon that monitors SMART. So if you don't want to manually monitor and run tests, you can set up smartd to run them on a regular basis. You should refer to ''man smartd'' and ''man smartd.conf'' and ''/etc/smartd.conf'' for everything you need to know about setting up smartd to do what you want it to do. ====== Sources ====== * [[https://en.wikipedia.org/wiki/S.M.A.R.T.|Wiki]] * [[https://www.us-cert.gov/security-publications/data-backup-options|CERT Data Backup Options]] * man smartctl {{tag>howtos template}}