Java Scanner Class: Reading Files, Strings and Parsing Tokens

The same file with the line 2 Java 1,5 true 1.6 is parsed differently by Scanner on different computers: with a German locale 1,5 becomes a double and 1.6 stays a string, with a US locale it is exactly the other way round. The program throws no exception, it just quietly produces a different result.

The java.util.Scanner class is a text scanner that splits its input into tokens using a delimiter (whitespace by default) and converts them into strings and primitive types using regular expressions. The source can be a file (File, Path), a byte stream (InputStream), a String, any Readable or a ReadableByteChannel.

This lesson treats Scanner as a parser of files and strings: charset, delimiters, pattern search, number locale, exceptions and closing the source. Reading from the keyboard, including the classic problem of nextLine() after nextInt(), is covered in Java User Input: Reading Data from Keyboard with System.in and Scanner.

1. How Scanner splits input into tokens

Scanner reads the source into its own buffer, skips delimiters and picks out a token: a piece of text between two delimiters. The hasNextXxx() methods check whether the next token fits a type without consuming it, and the nextXxx() methods consume the token and convert it to int, double, boolean or String.

The default delimiter is the regular expression \p{javaWhitespace}+: any run of spaces, tabs and line breaks. So the string "10   20\n30" yields exactly three tokens. The exceptions are nextLine(), findInLine(), findWithinHorizon() and skip(): they work on the raw text and ignore delimiters.

import java.util.Scanner;

public class TokenDemo {
    public static void main(String[] args) {
        try (Scanner scanner = new Scanner("10   20\n30 end")) {
            while (scanner.hasNextInt()) {
                System.out.println("Token: " + scanner.nextInt());
            }
            System.out.println("Token left: " + scanner.next());
        }
    }
}
// Output:
// Token: 10
// Token: 20
// Token: 30
// Token left: end

The loop stops at end: hasNextInt() returned false, but the token stays in the Scanner and next() picks it up.

2. Data sources and charset

A Scanner is created over a file, a stream, a string or a channel, and the source cannot be changed afterwards. For byte sources (a file, an InputStream, a channel) the charset matters: if it differs from the charset of the file, non-ASCII text turns into garbage or reading stops early. So it is better to pass the charset explicitly.

How File objects work is explained in Java File Class: Working with Files and Directories, and the Reader character streams are covered in Java Character Streams: Reader and Writer.

3. Reading a file line by line

To read a text file line by line, create a Scanner over a Path with an explicit charset inside try-with-resources and call nextLine() while hasNextLine() returns true. The nextLine() method returns the line without the line break, and an empty line of the file comes back as "".

Suppose the UTF-8 file users.txt contains three lines: Anna, an empty line and Peter.

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Path;
import java.util.Scanner;

public class ReadLinesDemo {
    public static void main(String[] args) throws IOException {
        Path file = Path.of("users.txt");                       // Java 11+
        try (Scanner scanner = new Scanner(file, StandardCharsets.UTF_8)) {
            int lineNumber = 0;
            while (scanner.hasNextLine()) {
                String line = scanner.nextLine();
                lineNumber++;
                if (line.isBlank()) {                           // Java 11+
                    continue;
                }
                System.out.println(lineNumber + ": " + line);
            }
            if (scanner.ioException() != null) {
                throw scanner.ioException();                    // see the section on exceptions
            }
        }
    }
}
// Output:
// 1: Anna
// 3: Peter
// If the file does not exist: java.nio.file.NoSuchFileException: users.txt

The resource in try (...) is closed automatically even if the loop throws an exception; the mechanism is explained in Try-with-resources in Java. If you need the whole file as a list of lines, Files.readAllLines(file) is shorter, and for very large files BufferedReader is faster (see Scanner vs BufferedReader vs Console). Scanner wins when a line has to be parsed into numbers and words right away.

4. Delimiters: useDelimiter() and delimiter()

The useDelimiter(String regex) method replaces the token delimiter with any regular expression, and delimiter() returns the current delimiter as a Pattern. The new delimiter fully replaces the whitespace one: if it does not contain \n, a line break becomes part of a token.

What does this code print?

import java.util.Scanner;

public class DelimiterQuiz {
    public static void main(String[] args) {
        try (Scanner scanner = new Scanner("Smith,35\nJones,28\n")) {
            scanner.useDelimiter(",");
            while (scanner.hasNext()) {
                System.out.println("[" + scanner.next() + "]");
            }
        }
    }
}

You would expect four tokens, but there are three, and two of them contain a line break:

[Smith]
[35
Jones]
[28
]

Calling nextInt() on the second token would throw InputMismatchException. The fix is to include both the comma and the end-of-line characters in the delimiter:

import java.util.Scanner;

public class DelimiterFixed {
    public static void main(String[] args) {
        try (Scanner scanner = new Scanner("Smith,35\nJones,28\n")) {
            scanner.useDelimiter("[,\\r\\n]+");
            System.out.println("Delimiter: " + scanner.delimiter());
            while (scanner.hasNext()) {
                String name = scanner.next();
                int age = scanner.nextInt();
                System.out.println(name + " - " + age);
            }
        }
    }
}
// Output:
// Delimiter: [,\r\n]+
// Smith - 35
// Jones - 28

Note the + quantifier. Without it, the delimiter "," on the string "a,,b" returns three tokens: "a", an empty string "" and "b". Sometimes that is what you want (an empty CSV field), but more often it is a source of surprising empty values. The reset() method restores the default delimiter \p{javaWhitespace}+.

5. Pattern search: findInLine, findWithinHorizon, skip

Besides splitting into tokens, Scanner can search the text for fragments that match a regular expression. findInLine() looks for a match up to the end of the current line, findWithinHorizon() looks within a given number of characters, skip() skips a match that starts exactly at the current position, and hasNext(String) checks whether the whole next token matches a pattern.

import java.util.Scanner;

public class FindDemo {
    public static void main(String[] args) {
        String text = "Order #1024 for 350 USD\nDelivery: 12.11.2026";
        try (Scanner scanner = new Scanner(text)) {
            System.out.println(scanner.findInLine("#(\\d+)"));          // #1024
            System.out.println(scanner.match().group(1));                // 1024
            System.out.println(scanner.findInLine("\\d+(?= USD)"));      // 350
            String date = "\\d{2}\\.\\d{2}\\.\\d{4}";
            System.out.println(scanner.findInLine(date));                // null
            System.out.println(scanner.findWithinHorizon(date, 0));      // 12.11.2026
        }

        try (Scanner scanner = new Scanner("ID-42 ID-7 X-1")) {
            while (scanner.hasNext("ID-\\d+")) {
                System.out.println(scanner.next("ID-\\d+"));            // ID-42, then ID-7
            }
            System.out.println("Stopped at: " + scanner.next());       // X-1
        }

        try (Scanner scanner = new Scanner("   >>> 15")) {
            System.out.println(scanner.skip("\\s*>>>").nextInt());      // 15
        }
    }
}

The date is not found by findInLine() because it is on the next line, while findWithinHorizon(date, 0) searches past the line break. skip() returns the same Scanner, so calls can be chained.

6. tokens() and findAll(): Scanner and the Stream API

Since Java 9 a Scanner can hand over its data as a Stream: tokens() returns a Stream<String> of the remaining tokens, and findAll(String regex) returns a Stream<MatchResult> with all matches of a pattern. This replaces the manual while (hasNext()) loop when tokens have to be filtered, converted or counted.

import java.util.List;
import java.util.Scanner;
import java.util.regex.MatchResult;
import java.util.stream.Collectors;

public class ScannerStreamDemo {
    public static void main(String[] args) {
        try (Scanner scanner = new Scanner("java  scanner\ttokens")) {
            List<String> words = scanner.tokens()
                    .map(String::toUpperCase)
                    .collect(Collectors.toList());
            System.out.println(words);                 // [JAVA, SCANNER, TOKENS]
        }

        try (Scanner scanner = new Scanner("Total: 120 USD, 45 USD, 300 USD")) {
            int total = scanner.findAll("\\d+")
                    .map(MatchResult::group)
                    .mapToInt(Integer::parseInt)
                    .sum();
            System.out.println(total);                 // 465
        }
    }
}

Until the terminal operation of the stream has finished, you must not call other methods of this Scanner: the stream and the Scanner share one read position. Calling tokens() or findAll() on a closed Scanner throws IllegalStateException.

7. Number bases and number locale

The numeric methods of Scanner depend on two settings: the radix (number base, 10 by default) and the locale, which defines the decimal separator and the digit-group separator. The radix is set by an argument such as nextInt(16) or by useRadix(), the locale by useLocale(); by default it is Locale.getDefault(Locale.Category.FORMAT).

import java.util.Scanner;

public class RadixDemo {
    public static void main(String[] args) {
        try (Scanner scanner = new Scanner("ff 1010 777 zz 0xff")) {
            System.out.println(scanner.nextInt(16));      // 255
            System.out.println(scanner.nextInt(2));       // 10
            scanner.useRadix(8);
            System.out.println(scanner.nextInt());        // 511
            System.out.println(scanner.nextInt(36));      // 1295
            System.out.println(scanner.hasNextInt(16));   // false: the 0x prefix is not supported
        }
    }
}

The locale is the reason for the story at the start of the lesson. Numbers in files, exports and logs are usually written with a dot, but Scanner parses them by the rules of the locale of the computer the program runs on:

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Locale;
import java.util.Scanner;

public class ScannerLocaleDemo {
    public static void main(String[] args) throws IOException {
        Path file = Path.of("scan.txt");
        Files.writeString(file, "2 Java 1,5 true 1.6");   // Java 11+, UTF-8
        parse(file, Locale.forLanguageTag("de-DE"));
        parse(file, Locale.US);
    }

    static void parse(Path file, Locale locale) throws IOException {
        System.out.println("Locale " + locale + ":");
        try (Scanner scanner = new Scanner(file, StandardCharsets.UTF_8)) {
            scanner.useLocale(locale);
            while (scanner.hasNext()) {
                if (scanner.hasNextInt()) {
                    System.out.println("  " + scanner.nextInt() + " : int");
                } else if (scanner.hasNextDouble()) {
                    System.out.println("  " + scanner.nextDouble() + " : double");
                } else if (scanner.hasNextBoolean()) {
                    System.out.println("  " + scanner.nextBoolean() + " : boolean");
                } else {
                    System.out.println("  " + scanner.next() + " : String");
                }
            }
        }
    }
}

Locale de_DE:
  2 : int
  Java : String
  1.5 : double
  true : boolean
  1.6 : String
Locale en_US:
  2 : int
  Java : String
  1,5 : String
  true : boolean
  1.6 : double

Another non-obvious point: the digit-group separator is localized too. With Locale.US the token "1,000" passes hasNextInt() and nextInt() returns 1000, although in a CSV file these could be two separate fields.

8. Scanner exceptions and ioException()

Scanner methods throw only unchecked exceptions: InputMismatchException when a token does not fit the type, NoSuchElementException when the input has ended, and IllegalStateException when the Scanner is already closed. Scanner does not propagate IOException; it remembers it, and you have to check it with ioException().

The most common InputMismatchException mistake is an infinite loop. The mismatched token is not consumed, so if you do not skip it in catch, the next nextInt() fails on the same token again:

import java.util.InputMismatchException;
import java.util.Scanner;

public class MismatchDemo {
    public static void main(String[] args) {
        int sum = 0;
        try (Scanner scanner = new Scanner("10 abc 20")) {
            while (scanner.hasNext()) {
                try {
                    sum += scanner.nextInt();
                } catch (InputMismatchException e) {
                    // without scanner.next() the loop never ends
                    System.out.println("Skipping: " + scanner.next());
                }
            }
        }
        System.out.println("Sum: " + sum);
    }
}
// Output:
// Skipping: abc
// Sum: 30

A swallowed IOException is more dangerous: a read failure looks like a normal end of file. For example, if a Scanner over a File with the UTF-8 charset meets bytes in windows-1252, the decoder throws MalformedInputException (a subclass of IOException), hasNextLine() returns false, and the program processes only the beginning of the file without reporting an error. That is why you should check scanner.ioException() after a file-reading loop, as in the example in Reading a file line by line.

9. close(): what is closed together with Scanner

The close() method closes not only the Scanner but also its source, if the source implements Closeable: a file, a Reader, an InputStream. So a Scanner over a file must be closed (preferably with try-with-resources), while a Scanner over System.in must not be closed in the middle of a program: standard input is closed with it and cannot be reopened.

import java.util.Scanner;

public class CloseSystemInDemo {
    public static void main(String[] args) {
        Scanner first = new Scanner(System.in);
        String name = first.nextLine();
        first.close();                         // also closes System.in

        Scanner second = new Scanner(System.in);
        String city = second.nextLine();       // NoSuchElementException: No line found
    }
}

The second Scanner is created without errors, but there is nothing left to read. Calling first.nextLine() again would throw a different exception: IllegalStateException: Scanner closed. What System.in is and why there is only one per program is explained in Java Standard Streams: System.in, System.out and System.err.

10. Scanner vs BufferedReader vs Console

Scanner is convenient when text has to be parsed into numbers and words right away; BufferedReader reads large amounts of lines faster but parses nothing; Console is meant for an interactive terminal and can read a password without echoing it. The choice depends on what matters more: convenient parsing, speed or talking to a user.

System.console() returns null when the program has no interactive terminal, for example when input is redirected from a file, so check the result before using it. The structure of BufferedReader is described in Java Character Streams: Reader and Writer.