Home Automation Log and Event Storage Calculator

Ask what a hub log costs in storage and the answer is almost always nothing worth worrying about. Ask what is in it and the answer is usually that one device wrote most of it. A power meter reporting every ten seconds files 8,640 records a day on its own, which is more than fifty contact sensors put together, and everything that reads the log pays for that whether or not the disk notices.

One group per line: name, how many devices, records a day each, bytes per record. A device that reports on a fixed interval writes 86400 divided by that interval — every 10 seconds is 8640, every minute is 1440. Blank lines and lines starting with # are ignored.
days
How long records are kept before they are purged or rolled up. This is a setting, not a fact about the software.
%
A database row is not just its payload — there are indexes, page headers and timestamps on top. A quarter is a reasonable working figure; measure your own if it matters.
:1
Set to 1 if nothing is compressed. Log records repeat themselves heavily so they compress well, but only where compression is actually switched on.
GB
The partition or card the log lives on, not the whole device.
Used to show how many rows a chart on the front page has to walk. This is usually the real cost of a chatty device, not the disk.
x
Bytes actually written to flash for every byte of data, once the filesystem and the card have had their turn. Used only for the wear estimate, and it is a guess unless you have measured it.
From the card or drive data sheet, if it gives one. Leave the wear line as a rough indicator; it is not a lifetime prediction and this page makes no claim about any product.
Automation Log Storage Calculator: Rows, Bytes, DaysBuildFigure

The disk is not the constraint, and saying so is the useful part

Run the default list and the answer is 11,248 records a day from 47 devices, about 2.25 MB of payload, and after a quarter added for indexes and 3.5:1 compression, roughly 0.8 MB a day stored. A year of that is around 294 MB. On a 32 GB card that is under one percent of the space, and the retention that card would allow runs to tens of thousands of days.

That is worth stating plainly because the question people usually ask is the storage one, and the honest answer is that a text log of home automation events is not a storage problem and almost never becomes one. If your log has become a problem, storage is not why.

What is actually expensive is the row count

Look at the split instead. Of those 11,248 daily records, 8,640 come from one energy monitor reporting every ten seconds. That is 77 percent of every row written by a single device, against 1,000 rows from twenty-five contact and motion sensors put together. Its payload share is similar — about 69 percent of the bytes.

The cost of that shows up wherever something reads the log rather than where something stores it. A seven-day chart on a dashboard walks roughly 78,700 rows to draw itself, and about 60,000 of them belong to the meter. Purges get slower, backups get bigger, and a restore takes proportionally longer. Delete that one device from the log and the annual row count drops from 4.11 million to about 950,000 — a 77 percent reduction from one configuration change, with no effect on anything except how finely you can see power.

The general shape holds well beyond this example. Devices that report on a fixed interval produce records whether or not anything happened; devices that report on change produce records only when something did. A house is full of the second kind and it takes exactly one of the first kind to outweigh all of them. Twenty-five sensors averaging 40 trips a day are a rounding error next to one meter on a ten-second timer.

The card, and the honest limits of the wear line

The wear indicator at the bottom is deliberately labelled as an order of magnitude. It multiplies the payload plus overhead by a write amplification factor and compares a year of that against the card capacity times its rated cycle count. On the defaults that comes out in the thousands of years, which is the correct conclusion for a text log and would be a completely different conclusion for a log holding image thumbnails or camera event clips.

Two reasons not to lean on it. Write amplification is unknowable without measuring, and a database doing frequent small synchronous writes can be far worse than the 4x default. And a rated cycle count is a manufacturer figure about a population, not a promise about your card. What the line is good for is a sanity check: if it comes out in years rather than centuries, something in the list is writing far more than a home automation log should.

Retention itself is a policy decision, not a technical one. The number that matters is not what fits, it is what you want kept, and that is the subject of the note about records of when a house was occupied.

Questions people ask

How much disk does a home automation history database need?

Far less than people expect. The default list here — 47 devices including one very chatty meter — comes to about 0.8 MB a day stored, so a year is under 300 MB. Unless something in your list is logging images, clips or full payloads of a few kilobytes each, the storage answer is almost always that it does not matter. The row count is the figure worth watching.

Why is my history slow when the database is small?

Because query cost follows rows, not bytes. A seven-day dashboard chart on the default list walks about 78,700 rows, roughly 60,000 of which come from one device reporting every ten seconds. Nothing about that is a disk problem, and adding disk does not help. Find the group with the biggest row share on this page and either lengthen its reporting interval or exclude it from the history.

What reporting interval should a power meter use?

That is a question about what you want to see rather than one this page answers, but the arithmetic is worth having in front of you first. Ten seconds is 8,640 records a day. One minute is 1,440. Five minutes is 288. Going from ten seconds to one minute cuts that device by 83 percent and, on the default list, cuts the whole log by 64 percent. If you only ever look at daily and monthly totals, you are paying for resolution nobody reads.

Does compression change the row count?

No, and that is the point of separating them on this page. Compression changes bytes on disk, which was not the problem. Rows stay exactly the same, so every query, purge, backup restore and chart draw costs exactly what it cost before. Compression is worth having and it fixes the thing that was already fine.

Should camera recordings be added to this list?

No. Video is a different order of magnitude and mixing it in makes both numbers useless. A single 4 Mbps camera stream is about 43 GB a day, which is roughly fifty thousand times this whole log. Size video separately on the camera storage retention calculator, and keep this page for the small structured records that come from sensors, switches and automations.

Related