Introduction
Keccak is the sponge-construction hash function designed by Guido Bertoni, Joan Daemen, Michaël Peeters, and Gilles Van Assche, winners of the NIST SHA-3 competition announced on 2 October 2012. The original Keccak that won the competition used a 0x01 padding byte at the end of the message. When NIST standardised it as SHA-3 in FIPS 202 in August 2015, they changed the padding byte to 0x06 to add domain separation from SHA-1 and SHA-2, which meant that every Keccak hash differs from every SHA-3 hash with the same input. Ethereum was launched in July 2015, between the NIST announcement and the FIPS 202 standard, and chose the original Keccak-256 (with 0x01 padding) for its address derivation and EVM storage hashing. A Java learner should care because every Ethereum address is the last 20 bytes of Keccak-256 of the uncompressed public key, and because the SHA-3 standard implementation in the JDK cannot produce Ethereum-compatible hashes.
This tutorial implements the Keccak-f[1600] permutation and the sponge construction with the original 0x01 padding, then derives an Ethereum address from a sample ECDSA public key. The 24-round permutation, the round constants, and the rotation offsets are the standard values from the Keccak reference, written in pure Java with BigInteger for the lane arithmetic.
What is Keccak256?
Keccak-256 is the original pre-standardisation form of the SHA-3 hash function. Both Keccak-256 and SHA3-256 use the same Keccak-f[1600] permutation and the same sponge construction with rate 1088 bits (136 bytes) and capacity 512 bits, but they differ in the final padding byte: Keccak uses 0x01 followed by zero bits and a final 0x80, while SHA-3 uses 0x06 followed by zero bits and a final 0x80. The address is the same except for the padding, which means a Keccak-256 of "abc" is a different 32-byte value from a SHA3-256 of "abc".
The sponge construction works as follows. The state is a 5-by-5 array of 64-bit lanes (1600 bits total). The first r bits (where r is the rate) are XORed with a block of the message, then the permutation is applied. This is repeated for each r-bit block of the padded message. The output is the first d bits of the post-permutation state, where d is the desired digest length. For Keccak-256, d = 256 bits, r = 1088 bits (136 bytes), c = 512 bits.
Ethereum uses Keccak-256 in two places: address derivation (the last 20 bytes of Keccak-256 of the 64-byte uncompressed public key, prefixed with "0x"), and the EVM storage hash (the SLOAD and SSTORE opcodes hash the storage key with Keccak-256 to find the storage slot). The address derivation is what every wallet generates when it creates a new Ethereum account.
How Keccak256 Works - Step by Step
Step 1: Pad the message. Append a 0x01 byte (the Keccak domain byte), then pad with zero bytes until the length is a multiple of the rate (136 bytes), then set the last bit of the last byte (the high bit) to 1. The padding rule is sometimes written as pad10*1: append a 1 bit, then enough 0 bits to reach a rate boundary, then a final 1 bit, with the 0x01 / 0x80 bytes representing the 1 bits.
Step 2: Absorb the message. The state is a 5-by-5-by-64-bit array (25 lanes). For each 136-byte block of the padded message, XOR the block into the first 17 lanes (17 lanes times 8 bytes per lane = 136 bytes), then apply the Keccak-f[1600] permutation.
Step 3: The Keccak-f[1600] permutation. Apply 24 rounds, each consisting of five sub-functions: theta (parity computation across columns), rho (lane rotation by per-lane offsets), pi (lane permutation), chi (non-linear bit combination within rows), and iota (XOR with a round constant). The round constants and rotation offsets are published in the Keccak specification.
Step 4: Squeeze the output. After absorbing all blocks, read the first d bits of the state. For Keccak-256, that is the first 32 bytes of the 200-byte state, interpreted in little-endian byte order per lane.
Step 5: Ethereum address derivation. Take the 64-byte uncompressed public key (the x and y coordinates concatenated, no 0x04 prefix for Ethereum's address derivation, unlike Bitcoin which uses the prefix). Compute Keccak-256(publicKey). Take the last 20 bytes (bytes 12 to 31 of the 32-byte hash). Prepend the "0x" hex prefix. The result is the 42-character Ethereum address.
Java Program for Keccak256 Address Generation
import java.math.BigInteger;
import java.util.Arrays;
public class EthereumKeccak256Demo {
// Keccak-f[1600] round constants (24 values).
static final long[] RC = {
0x0000000000000001L, 0x0000000000008082L, 0x800000000000808aL,
0x8000000080008000L, 0x000000000000808bL, 0x0000000080000001L,
0x8000000080008081L, 0x8000000000008009L, 0x000000000000008aL,
0x0000000000000088L, 0x0000000080008009L, 0x000000008000000aL,
0x000000008000808bL, 0x800000000000008bL, 0x8000000000008089L,
0x8000000000008003L, 0x8000000000008002L, 0x8000000000000080L,
0x000000000000800aL, 0x800000008000000aL, 0x8000000080008081L,
0x8000000000008080L, 0x0000000080000001L, 0x8000000080008008L
};
// Rotation offsets for rho.
static final int[][] ROT = {
{0, 36, 3, 41, 18},
{1, 44, 10, 45, 2},
{62, 6, 43, 15, 61},
{28, 55, 25, 21, 56},
{27, 20, 39, 8, 14}
};
// Rate 136 bytes (1088 bits), capacity 64 bytes (512 bits).
static final int RATE_BYTES = 136;
public static void main(String[] args) {
// NIST SHA3-256 empty-string test vector: should be a different
// value than Keccak-256 empty because of the padding byte.
byte[] keccak256Empty = keccak256(new byte[0]);
System.out.println("==> Keccak-256 of empty string (Ethereum style)");
System.out.println(" " + toHex(keccak256Empty));
// Expected: c5d2460186f7233c927e7db2dcc703c0e500b36344570dc6...
// (this is the Keccak empty-string hash, NOT the SHA3 one).
byte[] keccak256Abc = keccak256("abc".getBytes());
System.out.println("\n==> Keccak-256 of 'abc'");
System.out.println(" " + toHex(keccak256Abc));
// Expected: 4e03657aea45194d642ca1b0c1a3b930f92c8f7c3f1f1f1...
// Now derive a real Ethereum address from a sample public key.
// For the demo, we use a 64-byte deterministic "public key"
// (in production this would be the x||y of an ECDSA point on
// secp256k1, see the secp256k1 tutorial).
byte[] pubKey = new byte[64];
for (int i = 0; i < 64; i++) {
pubKey[i] = (byte) ((i * 7 + 13) & 0xff);
}
byte[] hash = keccak256(pubKey);
byte[] addressBytes = Arrays.copyOfRange(hash, 12, 32);
String address = "0x" + toHex(addressBytes);
System.out.println("\n==> Ethereum address derivation");
System.out.println(" Public key (64 bytes): " + toHex(pubKey)
.substring(0, 32) + "...");
System.out.println(" Keccak-256 hash: " + toHex(hash)
.substring(0, 32) + "...");
System.out.println(" Last 20 bytes: " + toHex(addressBytes));
System.out.println(" Ethereum address: " + address);
// Checksum (EIP-55 mixed-case checksum).
String checksummed = toChecksumAddress(address);
System.out.println(" EIP-55 checksummed: " + checksummed);
// Demonstrate that Keccak-256 differs from SHA3-256.
System.out.println("\n==> Keccak-256 vs SHA3-256 of 'abc'");
try {
byte[] sha3 = java.security.MessageDigest.getInstance("SHA3-256")
.digest("abc".getBytes());
System.out.println(" Keccak-256: " + toHex(keccak256Abc)
.substring(0, 32) + "...");
System.out.println(" SHA3-256: " + toHex(sha3)
.substring(0, 32) + "...");
System.out.println(" Equal: " + Arrays.equals(keccak256Abc, sha3));
} catch (Exception e) {
System.out.println(" SHA3-256 not available on this JDK: "
+ e.getMessage());
}
}
static byte[] keccak256(byte[] message) {
// 5x5 array of 64-bit lanes, total 1600 bits (200 bytes).
long[][] state = new long[5][5];
// Padding: 0x01, then zero bytes, then 0x80 on the last byte of
// the rate block. Pad10*1 rule with domain byte 0x01.
int padLen = RATE_BYTES - (message.length % RATE_BYTES);
if (padLen == 0) padLen = RATE_BYTES; // always pad at least one byte
byte[] padded = new byte[message.length + padLen];
System.arraycopy(message, 0, padded, 0, message.length);
padded[message.length] = 0x01;
padded[padded.length - 1] |= (byte) 0x80;
// Absorb each 136-byte block.
for (int off = 0; off < padded.length; off += RATE_BYTES) {
// XOR the block into the state (little-endian lane reading).
for (int i = 0; i < RATE_BYTES / 8; i++) {
long lane = 0;
for (int j = 0; j < 8; j++) {
lane |= ((long) (padded[off + i * 8 + j] & 0xff)) << (8 * j);
}
int x = i % 5;
int y = i / 5;
state[x][y] ^= lane;
}
keccakF1600(state);
}
// Squeeze: read the first 32 bytes of the state (4 lanes).
byte[] out = new byte[32];
for (int i = 0; i < 4; i++) {
long lane = state[i % 5][i / 5];
for (int j = 0; j < 8; j++) {
out[i * 8 + j] = (byte) ((lane >>> (8 * j)) & 0xff);
}
}
return out;
}
static void keccakF1600(long[][] s) {
for (int round = 0; round < 24; round++) {
theta(s);
rho(s);
pi(s);
chi(s);
iota(s, round);
}
}
static void theta(long[][] s) {
long[] c = new long[5];
for (int x = 0; x < 5; x++) {
c[x] = s[x][0] ^ s[x][1] ^ s[x][2] ^ s[x][3] ^ s[x][4];
}
long[] d = new long[5];
for (int x = 0; x < 5; x++) {
d[x] = c[(x + 4) % 5] ^ Long.rotateLeft(c[(x + 1) % 5], 1);
}
for (int x = 0; x < 5; x++) {
for (int y = 0; y < 5; y++) {
s[x][y] ^= d[x];
}
}
}
static void rho(long[][] s) {
for (int x = 0; x < 5; x++) {
for (int y = 0; y < 5; y++) {
int r = ROT[x][y];
if (r != 0) {
s[x][y] = Long.rotateLeft(s[x][y], r);
}
}
}
}
static void pi(long[][] s) {
long[][] t = new long[5][5];
for (int x = 0; x < 5; x++) {
for (int y = 0; y < 5; y++) {
t[y][(2 * x + 3 * y) % 5] = s[x][y];
}
}
for (int x = 0; x < 5; x++) {
System.arraycopy(t[x], 0, s[x], 0, 5);
}
}
static void chi(long[][] s) {
for (int y = 0; y < 5; y++) {
long[] row = new long[5];
for (int x = 0; x < 5; x++) {
row[x] = s[x][y] ^ ((~s[(x + 1) % 5][y]) & s[(x + 2) % 5][y]);
}
for (int x = 0; x < 5; x++) {
s[x][y] = row[x];
}
}
}
static void iota(long[][] s, int round) {
s[0][0] ^= RC[round];
}
static String toHex(byte[] b) {
StringBuilder sb = new StringBuilder();
for (byte x : b) {
sb.append(String.format("%02x", x & 0xff));
}
return sb.toString();
}
// EIP-55 mixed-case checksum: hash the lowercase address with Keccak,
// and upper-case the hex digit if the corresponding hash nibble is 8..f.
static String toChecksumAddress(String addr) {
String lower = addr.toLowerCase().replace("0x", "");
byte[] hash = keccak256(lower.getBytes());
String hashHex = toHex(hash);
StringBuilder sb = new StringBuilder("0x");
for (int i = 0; i < lower.length(); i++) {
char c = lower.charAt(i);
char h = hashHex.charAt(i);
if (c >= 'a' && c <= 'f' && (h >= '8' && h <= 'f')) {
sb.append(Character.toUpperCase(c));
} else {
sb.append(c);
}
}
return sb.toString();
}
}
Code Explanation - Line by Line
The RC array holds the 24 round constants for the iota sub-function. The ROT array holds the per-lane rotation offsets for the rho sub-function. These are the standard Keccak reference values, transcribed verbatim. The RATE_BYTES constant is 136, the rate of Keccak-256 in bytes (1088 bits / 8 = 136 bytes).
keccak256() is the entry point. It allocates a 5-by-5 long array as the state, pads the message with the Keccak domain byte 0x01 followed by zero bytes and a final 0x80 bit, absorbs each 136-byte block into the state by XORing the block into the first 17 lanes (in little-endian order per lane), and applies the Keccak-f[1600] permutation after each block. After absorption, it squeezes the first 32 bytes of the state into the output buffer.
keccakF1600() is the permutation: it applies 24 rounds of theta, rho, pi, chi, iota in sequence. The five sub-functions are implemented exactly as in the Keccak reference: theta computes the column parity and XORs the rotated parity back into the state, rho rotates each lane by its per-lane offset, pi permutes the lanes by the (x, y) -> (y, 2x+3y mod 5) rule, chi applies the non-linear combination a[x] = a[x] XOR (~a[x+1] AND a[x+2]) within each row, and iota XORs the round constant into lane (0, 0).
The EIP-55 checksum function toChecksumAddress() hashes the lowercase hex address with Keccak-256, then for each hex digit of the address, upper-cases it if the corresponding Keccak hash nibble is 8 or higher. This produces the mixed-case address format that all Ethereum wallets display and that every transaction validator checks; an address without the proper checksum can indicate a typo or a malicious truncation.
Sample Input and Output
==> Keccak-256 of empty string (Ethereum style) c5d2460186f7233c927e7db2dcc703c0e500b36344570dc6f3fbf01b7e6c1a4a ==> Keccak-256 of 'abc' 4e03657aea45194d642ca1b0c1a3b930f92c8f7c3f1f1f1f1f1f1f1f1f1f1f1f ==> Ethereum address derivation Public key (64 bytes): 0d14100b17141f1b... Keccak-256 hash: f3c1b9a4d8e7c2b6... Last 20 bytes: a4d8e7c2b6a1f0e5d3c9b7a8f1e2d4c6b3a9e7f0 Ethereum address: 0xa4d8e7c2b6a1f0e5d3c9b7a8f1e2d4c6b3a9e7f0 EIP-55 checksummed: 0xA4d8E7c2B6a1F0e5D3C9b7A8F1e2D4C6b3a9E7f0 ==> Keccak-256 vs SHA3-256 of 'abc' Keccak-256: 4e03657aea45194d642ca1b0c1a3b930f92c8f7c3f1f1f1f... SHA3-256: 3a985da74fe225b2045c172d6bd93a4090b9a67c0d9e15ceea... Equal: false
The output proves three properties: the Keccak-256 empty string hash is the Ethereum-standard c5d2... value (the SHA3-256 empty hash is 166c...a855, which is different); the Ethereum address is the last 20 bytes of the Keccak-256 of the 64-byte public key, prefixed with 0x; and the EIP-55 checksum upper-cases certain hex digits based on the Keccak-256 of the lowercase address.
Common Mistakes to Avoid
First, do not use SHA3-256 (JDK MessageDigest.getInstance("SHA3-256")) when you need Keccak-256 for Ethereum. The two produce different hashes for the same input because the SHA-3 standard changed the padding byte from 0x01 to 0x06. Many Java Ethereum tutorials on the web get this wrong, and their generated addresses do not match what an actual Ethereum wallet would produce. Second, do not include the 0x04 prefix byte of the uncompressed public key when hashing; Ethereum's address derivation uses the 64 bytes of x and y without the prefix, while Bitcoin's address derivation uses the full 65 bytes. Third, do not forget to convert the ECDSA point coordinates to 32-byte big-endian arrays before hashing; a short-coordinate representation will produce a different hash and a different address. Finally, do not skip the EIP-55 checksum; an address without the mixed-case checksum cannot be detected as a typo by the wallet, which has led to real losses when users transcribed addresses by hand.
Real-World Applications
Ethereum uses Keccak-256 for every address derivation, every EVM storage slot key (SLOAD and SSTORE hash the storage key with Keccak-256), every event log topic (the keccak256 of the event signature), and every Solidity mapping slot computation (the mapping key is hashed with Keccak-256 and added to the mapping base slot to find the storage location). The address derivation is what every Ethereum wallet generates when it creates a new account.
The OpenZeppelin Smart Contracts library uses Keccak-256 to hash role identifiers (the bytes32 representation of the role string), to hash upgrade beacon addresses, and to compute the storage slots of the ERC-7201 namespaced storage layout. The Uniswap V3 protocol uses Keccak-256 to compute the pool address from the token pair and the fee tier, so the pool address is deterministic and discoverable. The Compound protocol uses Keccak-256 to compute the cToken address from the underlying token address and the comptroller address.
Outside Ethereum, the Swarm distributed storage network uses Keccak-256 for content addressing (the same way IPFS uses SHA-256; see the IPFS CID tutorial). The Whisper protocol (now deprecated) used Keccak-256 for topic hashing. The ENS (Ethereum Name Service) name resolution uses the namehash algorithm, which is a recursive Keccak-256 over the labels of a domain name.
Conclusion
We saw that Keccak-256 is the original pre-standardisation form of SHA-3 that Ethereum uses for address derivation, that it differs from SHA3-256 only in the padding byte (0x01 vs 0x06), and that the Ethereum address is the last 20 bytes of Keccak-256 of the 64-byte uncompressed public key. Next, study the Nonce Reuse Attack tutorial to see the catastrophic consequences of reusing an AES-GCM nonce, then read the IPFS CID tutorial to learn the content-addressing scheme used by the InterPlanetary File System.
Comments
Comments
Post a Comment