Review Kenneth Nowak
This method uses a deep learning to produce an “opportunistic” blend forecast that leverages 5 different inputs. From training data, the systems assigns weights based on historical performance, as it pertains to current conditions. The method is fairly complex, but is generally well documented; I appreciated the diagrams/flow charts. I have no reservations about giving a “pass” rating on this one.
A couple questions for consideration:
- Has any analysis been done on the weights assigned to the inputs? Can any comment be made on the relative skill contributions of the various inputs? Were other potential inputs considered – if so, what and why not included?
- How did the weighting of the ECMWF EMOS vs ECMWF CNN play out? Was one more/less weighted? Did having two ECMWF inputs with different post-processing/calibration add value? If so, would it be possible/recommended to do same for NCEP and ECCC?
- In responding to Stephan, there is discussion that the 20-week rolling window might be large, but the benefit is that it helps guard against over-fitting, can comment be made on how 20-weeks was identified? I agree, seems large; would have expected something more around ~10 weeks as a “seasonal” rolling window. From the documentation, it sounds like there may have been different configurations tested/tried. What if anything can be shared about those configurations and if they played into the 20-week selection?