Making REST Calls Retryable in Biking Weather Suitability Forecast Application

One frustration I had noticed with my Biking Weather Suitability Forecast Application was that I found it often would not show results until I reloaded the page, and a look at the application logs showed that the REST calls to one of the APIs called had failed or timed out. After doing some research into using Spring’s Retryable options for methods, I decided to make the DailyReportCollectionService method getCurrentDailyReports() retryable, so that it would be attempted a second time after a one second pause if the first try didn’t succeed in returning complete data, with a third and final attempt made as the “Recover” option.

First, I added the following two required dependencies to the dependencies section of the application’s pom.xml file.

        <dependency>
            <groupId>org.springframework.retry</groupId>
            <artifactId>spring-retry</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-aspects</artifactId>
        </dependency>

I needed to add a new Exception, IncompleteForecastException, which would trigger retries of the failed method call. I created a new “exception” package, with the following package-info.java to document the package.

/**
 * Package containing custom Exceptions for biking weather application.
 */
package biz.noip.johnwatne.bikingweather.exception;

The IncompleteForecastException class is a very simple extension of the Exception class, with the following code.

package biz.noip.johnwatne.bikingweather.exception;

/**
 * Exception thrown if the forecast information returned has either less than
 * seven days' data, or missing sunrise or sunset information.
 *
 * @author John Watne
 *
 */
public class IncompleteForecastException extends Exception {
    private static final long serialVersionUID = 1L;

    public IncompleteForecastException(final String message) {
        super(message);
    }
}

I then made the changes required to have the retryable code in DailyReportCollectionService. Since the existing getCurrentDailyReports() needed to be executed both as the standard Retryable code and as the Recovery code, it’s logic was moved to a new private method, appropriately named getRetryableCurrentDailyReports(), starting as follows.

    /**
     * The shared code of {@link #getCurrentDailyReports()} and
     * {@link #getFInalCurrentDailyReports(IncompleteForecastException)}.
     *
     * @return the latest {@link DailyReportCollection}.
     */
    private DailyReportCollection getRetryableCurrentDailyReports() {
.
.
.
.

I added a couple String constants for possible error return values passed on from getRetryableCurrentDailyReports().

    private static final String MISSING_SUNSET_DATA_FOR_TIME =
            "Missing sunset data for time ";
    private static final String MISSING_SUNRISE_DATA_FOR_TIME =
            "Missing sunrise data for time ";
    private static final String MISSING_FORECAST_DATA = "Missing forecast data";
    private static final String NO_DAILY_REPORT_COLLECTION_OBTAINED =
            "No DailyReportCollection obtained";

The existing getCurrentDailyReports() method was revised to be a Retryable caller of the extracted getRetryableCurrentDailyReports(), throwing an IncompleteForecastException with one of the above messages when needed.

    /**
     * Makes calls to get and return the latest {@link DailyReportCollection}.
     * If forecast data is incomplete, throw an
     * {@link IncompleteForecastException} and attempt again in a second.
     *
     * @return the latest {@link DailyReportCollection}.
     * @throws IncompleteForecastException
     *             if the DailyReportCollection to be returned is incomplete.
     */
    @Retryable(value = {IncompleteForecastException.class}, maxAttempts = 2,
            backoff = @Backoff(delay = 1000))
    public DailyReportCollection getCurrentDailyReports()
            throws IncompleteForecastException {
        final DailyReportCollection reports = getRetryableCurrentDailyReports();

        if (reports == null) {
            LOGGER.warn(NO_DAILY_REPORT_COLLECTION_OBTAINED);
            throw new IncompleteForecastException(
                    NO_DAILY_REPORT_COLLECTION_OBTAINED);
        } else {
            final SortedMap<Long,
                    DailyAndHourlySuitabilityForDay> dailyReports =
                            reports.getDailyReports();

            if (dailyReports.size() < 7) {
                LOGGER.warn(MISSING_FORECAST_DATA);
                throw new IncompleteForecastException(MISSING_FORECAST_DATA);
            }

            for (Entry<Long,
                    DailyAndHourlySuitabilityForDay> entry : dailyReports
                            .entrySet()) {
                final DailyAndHourlySuitabilityForDay suitabilityForDay =
                        entry.getValue();
                final Long dayValue = entry.getKey();

                if (!StringUtils.hasText(suitabilityForDay.getSunrise())) {
                    LOGGER.warn(MISSING_SUNRISE_DATA_FOR_TIME + dayValue);
                    throw new IncompleteForecastException(
                            MISSING_SUNRISE_DATA_FOR_TIME + dayValue);
                }

                if (!StringUtils.hasText(suitabilityForDay.getSunset())) {
                    LOGGER.warn(MISSING_SUNSET_DATA_FOR_TIME + dayValue);
                    throw new IncompleteForecastException(
                            MISSING_SUNSET_DATA_FOR_TIME + dayValue);
                }
            }
        }

        return reports;
    }

I then added the new recovery method, getFInalCurrentDailyReports(final IncompleteForecastException e) [sic], making the third and final attempt to get results, if needed.

    /**
     * Final attempt at getting complete data for the
     * {@link DailyReportCollection} that was to be returned by
     * {@link #getCurrentDailyReports()}. This time, do not check for
     * completeness of data, and just return results obtained.
     *
     * @param e
     *            the last {@link IncompleteForecastException} thrown by the
     *            last retry of {@link #getCurrentDailyReports()}.
     * @return the latest {@link DailyReportCollection}.
     */
    @Recover
    public DailyReportCollection
            getFInalCurrentDailyReports(final IncompleteForecastException e) {
        LOGGER.warn(
                "Final attempt at getting current daily reports - not checking for completeness afterward!");
        return getRetryableCurrentDailyReports();
    }

With the additional information provided in the message contained within any IncompleteForecastException thrown, the ForecastController class’ getForecast(model) method was slightly modified to write to the error log if the call to the DailyReportCollectionService’s getCurrentDailyReports() method resulted in such an Exception being thrown.

    @GetMapping("/")
    public String getForecast(final Model model) {
        model.addAttribute("cityname", cityName);

        try {
            model.addAttribute("dailyReports",
                    dailyReportCollectionService.getCurrentDailyReports());
        } catch (IncompleteForecastException e) {
            LOGGER.error("Errored out of all attempts to get forecast", e);
        }

        return "forecast";
    }

Finally, the EnableRetry annotation was added to the BikingweatherApplication class as part of the configuration required by the application to retry any Retryable code.

@EnableRetry
@SpringBootApplication(exclude = {DataSourceAutoConfiguration.class})
@ComponentScan(basePackageClasses = {ForecastController.class,
        DailyAndHourlySuitabilityForDay.class,
        DailyReportCollectionService.class, LoggingAspect.class})
public class BikingweatherApplication extends SpringBootServletInitializer {
.
.
.
.
}

After making these additions, I have found that needing to reload the page to see results occurs much less frequently. I hope others may find this retryable code capability valuable.

Website FINALLY Adapted to Apple Silicon

Back in March, I took advantage of a sale at Costco, advanced the inevitable update, and bought a blue-colored iMac. With its Apple M1 chip, after reinstalling essential packages using Homebrew, I had to make multiple changes to the Apache and Tomcat server configurations to allow my website, and the Tomcat server serving the biking weather suitability forecast application, to work properly again. Since these explorations – including a dead end trying to switch from using Apache to Nginx for the web server, given what seems to be the trend and the latter’s apparent strength being reverse proxy work – were done very sporadically, it took me until this month finally to get it right.

One thing I am glad I did relatively early in the process was to make the new configuration folders git repositories, so I could review the history of document changes and, more importantly, reverse them if needed. This was done using the following steps for the Apache (httpd) folder:

cd /opt/homebrew/etc/httpd/
git init
git checkout -b starting_m1_config
git add --all
git commit -m 'Initial commit of starting M1 config'

The first part of the update was dealing with the change of all /user/local/etc/httpd paths to /opt/homebrew/etc/httpd. After copying my backed-up configuration files from /usr/local/etc/httpd to /opt/homebrew/etc/httpd, I did a global search and replace for this path change throughout the files in the updated folder.

I uncommented a few modules and updated the php_module path reference in httpd.conf:

LoadModule xml2enc_module /opt/homebrew/Cellar/httpd/2.4.53/lib/httpd/modules/mod_xml2enc.so
LoadModule proxy_html_module /opt/homebrew/Cellar/httpd/2.4.53/lib/httpd/modules/mod_proxy_html.so
.
.
.
LoadModule proxy_connect_module /opt/homebrew/Cellar/httpd/2.4.53/lib/httpd/modules/mod_proxy_connect.so
.
.
.
LoadModule php_module /opt/homebrew/Cellar/php/8.1.6/lib/httpd/modules/libphp.so

Finally, I moved the ProxyPass and ProxyPassReverse parameters from httpd.conf to the two <VirtualHost *:443> elements – the only difference between the two being one with a ServerName of localhost, the other johnwatne.no-ip.biz – within the extra/httpd-vhosts.conf configuration file. I also added additional proxy-related parameters based on suggestions in various items found in searching for answers. Whether or not they were needed, I don’t know, but they are part of my final, working solution:

   ProxyPreserveHost Off
   ProxyRequests Off
   ProxyVia Off
   <Proxy *>
      Require all granted
   </Proxy>
   ProxyPass "/bikingweather" "http://localhost:[tomcat-port]/bikingweather"
   ProxyPassReverse "/bikingweather" "http://localhost:[tomcat-port]/bikingweather"

In the code above, [tomcat-port] is the port on which the Tomcat server is listening, which is 8080 by default. The final thing I needed to do with the Apache configuration was to make sure I was consistent in having both paths listed for the ProxyPass and ProxyPassReverse parameters not end with a “/” character. Having only one of them end with the additional character was breaking the proxy URL reference, and I kept getting whitelabel error pages from Tomcat.

One item I needed to fix for the update to Apple Silicon on the web application side was to update one of the dependencies used by the biking suitability forecast application, a Spring Boot application. Looking through the logs for the application when loading the application, I saw the following [edited to remove more info than is likely helpful]:

2022-05-29 23:04:44,238 ERROR nsServerAddressStreamProviders[line  73] Unable to load io.netty.resolver.dns.macos.MacOSDnsServerAddressStreamProvider, fallback to system defaults. This may result in incorrect DNS resolutions on MacOS.
java.lang.reflect.InvocationTargetException: null
	at jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:79) ~[?:?]
	at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) ~[?:?]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:483) ~[?:?]
	at io.netty.resolver.dns.DnsServerAddressStreamProviders.<clinit>(DnsServerAddressStreamProviders.java:64) ~[netty-resolver-dns-4.1.76.Final.jar:4.1.76.Final]
.
.
.
Caused by: java.lang.UnsatisfiedLinkError: failed to load the required native library
	at io.netty.resolver.dns.macos.MacOSDnsServerAddressStreamProvider.ensureAvailability(MacOSDnsServerAddressStreamProvider.java:110) ~[netty-resolver-dns-classes-macos-4.1.76.Final.jar:4.1.76.Final]
	at io.netty.resolver.dns.macos.MacOSDnsServerAddressStreamProvider.<init>(MacOSDnsServerAddressStreamProvider.java:120) ~[netty-resolver-dns-classes-macos-4.1.76.Final.jar:4.1.76.Final]
	at jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67) ~[?:?]
	... 94 more
Caused by: java.lang.UnsatisfiedLinkError: could not load a native library: netty_resolver_dns_native_macos_aarch_64
	at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:224) ~[netty-common-4.1.76.Final.jar:4.1.76.Final]
.
.
.
	... 91 more
	Suppressed: java.lang.UnsatisfiedLinkError: could not load a native library: netty_resolver_dns_native_macos
		at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:224) ~[netty-common-4.1.76.Final.jar:4.1.76.Final]
.
.
.
	Caused by: java.io.FileNotFoundException: META-INF/native/libnetty_resolver_dns_native_macos.jnilib
		at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:166) ~[netty-common-4.1.76.Final.jar:4.1.76.Final]
		... 99 more
		Suppressed: java.lang.UnsatisfiedLinkError: no netty_resolver_dns_native_macos in java.library.path: /Users/John/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
			at java.lang.ClassLoader.loadLibrary(ClassLoader.java:2434) ~[?:?]
.
.
.
Caused by: java.io.FileNotFoundException: META-INF/native/libnetty_resolver_dns_native_macos_aarch_64.jnilib
	at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:166) ~[netty-common-4.1.76.Final.jar:4.1.76.Final]
.
.
.

A little bit of digging revealed that these were related to using the Intel version of the netty-resolver-dns-native-macos Maven artifact. To use the correct version, I needed to add it to the dependencies, along with the dependency-specific “classifier” value to ensure I was using the Apple Silicon – aarch_64 – version of the resolver. This was done by adding the following to the list of dependencies in the application’s pom.xml file.

        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-resolver-dns-native-macos</artifactId>
            <classifier>osx-aarch_64</classifier>
        </dependency>

I also updated the netty.version value in the properties section of pom.xml to the current version.

After all these fixes, the main WordPress site’s and the web application’s links to each other – including shared font files – are once again working correctly.

Adding a Dark Theme to the Biking Weather Suitability Forecast Application

Having read a few articles online about offering a dark theme as part of improving accessibility for certain people, I decided to add a dark theme option both for my website overall, and for the Biking Weather Suitability Forecast application. I will discuss what steps I took to add this to my web application; similar changes needed to be made for overrides to the default theme CSS and script files for the general website.
The easiest thing to do was to download the dark version of the “Powered by Dark Sky” icon. This will likely need to be removed in a few months, after Dark Sky stops their “grandfathered” API support at the end of 2022.
Next, I added some overrides at the bottom of my main.css file for the case when the user-specified preferred color scheme is dark, such as when choosing “dark mode” on an iPhone or Mac.

/* Dark mode color scheme */
@media(prefers-color-scheme: dark) {
	body, .card, .table, caption {
		background-color: black;
		color: white;
	}
	
	td.status0, td.status1, td.status2, td.status3 {
		color: black;
	}
}

To allow writing a script that would allow changing the “Powered by Dark Sky” graphic to one with inverted colors, I added the “darksky” class to the img tag in the forecast.html page:

		<a href="https://darksky.net/poweredby">
 <img class="darksky" th:src="@{/img/poweredby.png}" alt="Powered by Dark Sky" style="height: 3rem;" />
</a>

Finally, I used jQuery and plain JavaScript to check if the dark mode was the preferred color scheme both on page load and on a change to the color scheme preference. If so, I called the “darkMode()” function shown below to change the image. Otherwise, if changed back to the default light mode, the default “bright” icon is restored. This was done by adding the following script to the bottom of forecast.html, after the default jquery, bootstrap, and bootstrap script includes.

	<script>
		function darkMode() {
			$("img.darksky").attr("src", 
					"https://johnwatne.no-ip.biz/bikingweather/img/poweredby-darkbackground.png");
		}
		
		var mql = window.matchMedia('(prefers-color-scheme: dark)');
		
		if (mql.matches) {
			darkMode();
		}
		
		window.matchMedia('(prefers-color-scheme: dark)').addEventListener('change', event => {
			if (event.matches) {
				darkMode();
			} else {
				// Restore standard / light mode image.
				$("img.darksky").attr("src", 
						"https://johnwatne.no-ip.biz/bikingweather/img/poweredby.png");
			}
		})
	</script>

The following images show examples of what the page now looks like in both the default light and optional dark mode. I will note that I need to fix the contrast on the linked text in dark mode. This will be an additional override that I will add soon to the CSS file.

Biking Weather Suitability Forecast application, shown in default light mode
Biking Weather Suitability Forecast application, shown in default light mode
Biking Weather Suitability Forecast application, shown in dark mode
Biking Weather Suitability Forecast application, shown in dark mode