Making REST Calls Retryable in Biking Weather Suitability Forecast Application

One frustration I had noticed with my Biking Weather Suitability Forecast Application was that I found it often would not show results until I reloaded the page, and a look at the application logs showed that the REST calls to one of the APIs called had failed or timed out. After doing some research into using Spring’s Retryable options for methods, I decided to make the DailyReportCollectionService method getCurrentDailyReports() retryable, so that it would be attempted a second time after a one second pause if the first try didn’t succeed in returning complete data, with a third and final attempt made as the “Recover” option.

First, I added the following two required dependencies to the dependencies section of the application’s pom.xml file.

        <dependency>
            <groupId>org.springframework.retry</groupId>
            <artifactId>spring-retry</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-aspects</artifactId>
        </dependency>

I needed to add a new Exception, IncompleteForecastException, which would trigger retries of the failed method call. I created a new “exception” package, with the following package-info.java to document the package.

/**
 * Package containing custom Exceptions for biking weather application.
 */
package biz.noip.johnwatne.bikingweather.exception;

The IncompleteForecastException class is a very simple extension of the Exception class, with the following code.

package biz.noip.johnwatne.bikingweather.exception;

/**
 * Exception thrown if the forecast information returned has either less than
 * seven days' data, or missing sunrise or sunset information.
 *
 * @author John Watne
 *
 */
public class IncompleteForecastException extends Exception {
    private static final long serialVersionUID = 1L;

    public IncompleteForecastException(final String message) {
        super(message);
    }
}

I then made the changes required to have the retryable code in DailyReportCollectionService. Since the existing getCurrentDailyReports() needed to be executed both as the standard Retryable code and as the Recovery code, it’s logic was moved to a new private method, appropriately named getRetryableCurrentDailyReports(), starting as follows.

    /**
     * The shared code of {@link #getCurrentDailyReports()} and
     * {@link #getFInalCurrentDailyReports(IncompleteForecastException)}.
     *
     * @return the latest {@link DailyReportCollection}.
     */
    private DailyReportCollection getRetryableCurrentDailyReports() {
.
.
.
.

I added a couple String constants for possible error return values passed on from getRetryableCurrentDailyReports().

    private static final String MISSING_SUNSET_DATA_FOR_TIME =
            "Missing sunset data for time ";
    private static final String MISSING_SUNRISE_DATA_FOR_TIME =
            "Missing sunrise data for time ";
    private static final String MISSING_FORECAST_DATA = "Missing forecast data";
    private static final String NO_DAILY_REPORT_COLLECTION_OBTAINED =
            "No DailyReportCollection obtained";

The existing getCurrentDailyReports() method was revised to be a Retryable caller of the extracted getRetryableCurrentDailyReports(), throwing an IncompleteForecastException with one of the above messages when needed.

    /**
     * Makes calls to get and return the latest {@link DailyReportCollection}.
     * If forecast data is incomplete, throw an
     * {@link IncompleteForecastException} and attempt again in a second.
     *
     * @return the latest {@link DailyReportCollection}.
     * @throws IncompleteForecastException
     *             if the DailyReportCollection to be returned is incomplete.
     */
    @Retryable(value = {IncompleteForecastException.class}, maxAttempts = 2,
            backoff = @Backoff(delay = 1000))
    public DailyReportCollection getCurrentDailyReports()
            throws IncompleteForecastException {
        final DailyReportCollection reports = getRetryableCurrentDailyReports();

        if (reports == null) {
            LOGGER.warn(NO_DAILY_REPORT_COLLECTION_OBTAINED);
            throw new IncompleteForecastException(
                    NO_DAILY_REPORT_COLLECTION_OBTAINED);
        } else {
            final SortedMap<Long,
                    DailyAndHourlySuitabilityForDay> dailyReports =
                            reports.getDailyReports();

            if (dailyReports.size() < 7) {
                LOGGER.warn(MISSING_FORECAST_DATA);
                throw new IncompleteForecastException(MISSING_FORECAST_DATA);
            }

            for (Entry<Long,
                    DailyAndHourlySuitabilityForDay> entry : dailyReports
                            .entrySet()) {
                final DailyAndHourlySuitabilityForDay suitabilityForDay =
                        entry.getValue();
                final Long dayValue = entry.getKey();

                if (!StringUtils.hasText(suitabilityForDay.getSunrise())) {
                    LOGGER.warn(MISSING_SUNRISE_DATA_FOR_TIME + dayValue);
                    throw new IncompleteForecastException(
                            MISSING_SUNRISE_DATA_FOR_TIME + dayValue);
                }

                if (!StringUtils.hasText(suitabilityForDay.getSunset())) {
                    LOGGER.warn(MISSING_SUNSET_DATA_FOR_TIME + dayValue);
                    throw new IncompleteForecastException(
                            MISSING_SUNSET_DATA_FOR_TIME + dayValue);
                }
            }
        }

        return reports;
    }

I then added the new recovery method, getFInalCurrentDailyReports(final IncompleteForecastException e) [sic], making the third and final attempt to get results, if needed.

    /**
     * Final attempt at getting complete data for the
     * {@link DailyReportCollection} that was to be returned by
     * {@link #getCurrentDailyReports()}. This time, do not check for
     * completeness of data, and just return results obtained.
     *
     * @param e
     *            the last {@link IncompleteForecastException} thrown by the
     *            last retry of {@link #getCurrentDailyReports()}.
     * @return the latest {@link DailyReportCollection}.
     */
    @Recover
    public DailyReportCollection
            getFInalCurrentDailyReports(final IncompleteForecastException e) {
        LOGGER.warn(
                "Final attempt at getting current daily reports - not checking for completeness afterward!");
        return getRetryableCurrentDailyReports();
    }

With the additional information provided in the message contained within any IncompleteForecastException thrown, the ForecastController class’ getForecast(model) method was slightly modified to write to the error log if the call to the DailyReportCollectionService’s getCurrentDailyReports() method resulted in such an Exception being thrown.

    @GetMapping("/")
    public String getForecast(final Model model) {
        model.addAttribute("cityname", cityName);

        try {
            model.addAttribute("dailyReports",
                    dailyReportCollectionService.getCurrentDailyReports());
        } catch (IncompleteForecastException e) {
            LOGGER.error("Errored out of all attempts to get forecast", e);
        }

        return "forecast";
    }

Finally, the EnableRetry annotation was added to the BikingweatherApplication class as part of the configuration required by the application to retry any Retryable code.

@EnableRetry
@SpringBootApplication(exclude = {DataSourceAutoConfiguration.class})
@ComponentScan(basePackageClasses = {ForecastController.class,
        DailyAndHourlySuitabilityForDay.class,
        DailyReportCollectionService.class, LoggingAspect.class})
public class BikingweatherApplication extends SpringBootServletInitializer {
.
.
.
.
}

After making these additions, I have found that needing to reload the page to see results occurs much less frequently. I hope others may find this retryable code capability valuable.

Adding Verbal Warnings and Time Quotas Configurable by Account for Forced Logouts in Programmatic MacOS Parental Controls

Building upon the original Restoring Forced Logouts Removed from MacOS Parental Controls, the original refinements, and the New Year updates to them, I have since added code to add the following features.

  1. Give two verbal warnings about (approximate) time remaining before forced logout; and
  2. Make the minutes allowed per day a configurable property which can be customized for individual accounts.

The first was added as a courtesy to our boys. The second was added to allow cutting back or eliminating hours for each boy individually if they didn’t complete their list of basic daily tasks to be done to be allowed the full amount of computer time for the following day, or to increase it for sick days and the like. Making it a property also allows changing it on the fly, without having to alter the code, and also eliminates some very inconvenient hard-coded values.

The properties were added to the end of the application.properties file. At the same time, I added a missing JPA hibernate dialect property that was needed to get the application to work properly with my MariaDB instance.

spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQLDialect
# Map properties
users.minutes.[kid-login-1]=60
users.minutes.[kid-login-2]=60

Here, [kid-login-1] and [kid-login-2] are the login ID’s for our boys. The default (for us) values of 60 minutes are set for both accounts.

These properties are mapped to a new ConfigProperties.java class in the model package for the application, which looks like the following.

package biz.noip.johnwatne.logtimer.model;

import java.util.Map;

import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;

@Configuration
@ConfigurationProperties(prefix = "users")
public class ConfigProperties {
    private Map<String, Long> minutes;

    public Map<String, Long> getMinutes() {
        return minutes;
    }

    public void setMinutes(Map<String, Long> minutes) {
        this.minutes = minutes;
    }
}

We see that the “users” prefix in the “ConfigurationProperties” annotation for the class indicates that those properties in the application.properties file that start with “users.” specify values to use within this configuration class. The configuration class contains a single attribute, “minutes” which maps String values for login IDs to Long values for minutes per day allowed. This ties to the users.minutes.[username]=[minutes] properties in the file. The username part of the property is added as a key to the Map, and the value assigned to the property is set to the corresponding value in the Map.

With the new ConfigProperties class and corresponding properties added, the remaining changes were made to the LogtimerRunner class.

First, the ConfigProperties bean needed to be added as an attribute in LogtimerRunner as follows:

    @Autowired
    private ConfigProperties configProperties;

Then, the logoutIfKidLoggedInTooLong method was modified to

  1. replace the switch statement with hard-coded cases for each child’s name with an iteration of all properties defined in ConfigProperties.getMinutes(), and
  2. Add a new call to the standard MacOS “say” command to notify of time left when there are 10 or fewer minutes left until the forced logout.

The updated method is as follows.

    /**
     * Logs out the specified user if
     * <ol>
     * <li>they are a named child within the family, and</li>
     * <li>the number of minutes they have been logged in exceeds the maximum
     * limit.</li>
     * </ol>
     *
     * @param user
     *            the user whose login time is being checked.
     * @param minutes
     *            the number of minutes the user has been logged in.
     * @throws IOException
     *             if an I/O error occurs.
     * @throws InterruptedException
     *             if an thread is interrupted.
     */
    public void logoutIfKidLoggedInTooLong(final String user,
            final long minutes) throws IOException, InterruptedException {
        if (configProperties != null) {
            final Map<String, Long> map = configProperties.getMinutes();

            if (map != null) {
                Long maxMinutes = map.getOrDefault(user, null);

                if (maxMinutes != null) {
                    LOGGER.debug("maxMinutes for " + user + ": " + maxMinutes
                            + "; minutes: " + minutes);

                    if (minutes > Math.max(maxMinutes - 1, 0)) {
                        LOGGER.info("Logging out user " + user);
                        ProcessBuilder processBuilder = new ProcessBuilder(
                                "/bin/sh", "-c", "./logout-user.sh " + user);
                        processBuilder.directory(new File("/Users/John"));
                        Process process = processBuilder.start();

                        try (final BufferedReader reader =
                                new BufferedReader(new InputStreamReader(
                                        process.getInputStream()))) {
                            String line;

                            while ((line = reader.readLine()) != null) {
                                LOGGER.info(line);
                            }

                            LOGGER.debug(Integer.toString(process.waitFor()));
                        }
                    } else if (minutes > Math.max(maxMinutes - 11, 0)) {
                        final long minutesToGo = maxMinutes - minutes;
                        final String notification = "Logging out user " + user
                                + " in about " + minutesToGo + " minutes";
                        LOGGER.info(notification);
                        ProcessBuilder processBuilder = new ProcessBuilder(
                                "/bin/sh", "-c", "say '" + notification + "'");
                        Process process = processBuilder.start();
                        LOGGER.debug(Integer.toString(process.waitFor()));
                    }
                } else {
                    LOGGER.debug("Time limits not enforced for user " + user);
                }
            }
        } else {
            LOGGER.error("ERROR!! UNABLE TO READ CONFIGURATION PROPERTIES");
        }
    }

I also fixed an off-by-one error in checking the count of login processes for each user in the following method. The value of getCountOfLoginProcessesForUser(persistedLogin.getUsername()) is now checked if < 1, rather than < 2.

    /**
     * Sets the logout time for users no longer logged in, if not already set,
     * to the specified logout time.
     *
     * @throws IOException
     *             if an I/O error occurs.
     */
    private void setLogoutTimesForUsersNotLoggedIn() throws IOException {
        // Do final pass, setting logout times for users no longer logged in.
        for (Logins persistedLogin : loginsRepository.findAll()) {
            if ((persistedLogin.getLogout() == null)
                    && (getCountOfLoginProcessesForUser(
                            persistedLogin.getUsername()) < 1)) {
                // User logged out but not yet indicated as such, so set logout
                // time to now.
                LOGGER.info("*** " + persistedLogin.getUsername()
                        + " logged out; setting logout to now");
                persistedLogin.setLogout(LocalDateTime.now());
                loginsRepository.save(persistedLogin);
                LOGGER.info("*** Record updated: " + persistedLogin);
            }
        }
    }

Finally, to reduce the amount of logging by eliminating info-level logging I had used during initial development, I changed the calculation of lines read from the BufferedReader for the results of the call to get the count of login processes for user in the method getCountOfLoginProcessesForUser(final String user). I removed the logging of each line of output by simplifying

lines = reader.lines().peek(e -> LOGGER.info(e)).count();

to

lines = reader.lines().count();

I hope there might be others who might find these changes to add flexibility to the program helpful. I may consider adding this to GitHub at some point.