\documentclass[10pt]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage[hmargin=1.2in, vmargin=1.2in]{geometry}
\usepackage{amsmath,amsfonts}

\title{\large Review of \emph{On Bayes and Nash experiment design for hypothesis testing problems}}
\date{}

\begin{document}

\maketitle

\paragraph{Summary.} This paper studies a standard ANOVA hypothesis testing
problem: given $K$ treatments, determine whether they all have the same average
effect, or whether there exists a pair of treatments whose average effects are
different. The authors consider the likelihood ratio test and the focus is on
finding and characterizing the optimal design.  The contributions are as
follows:
\begin{itemize}
	\item first, optimality is formulated as a maxi-min problem: find the most
		powerful design for the worst possible value of the unknown treatment
		effects in the alternative hypothesis.  A previous paper of the authors
		had established that the balanced design is optimal for this criterion.
	\item next, a ``pseudo-Bayes'' approach is adopted: there is a prior on the
		average treatment effects, and the goal is to find the design which is
		most powerful on average over this prior. It is found that the balanced
		design is optimal in that sense whenever the prior is
		exchangeable.
	\item next, the authors consider a game theoretic approach where the
		optimal design problem is formulated as a two-player zero-sum game
		between the ``max'' player attempting to find the optimal design and
		the ``min'' player attempting to find the worst possible configuration
		of average effects. The unique Nash equilibrium of the game is computed
		for two different choices of strategy space for the max player. Again,
		the balanced design is found to be the optimal strategy (assuming it is
		contained in the strategy space of the max player).
	\item finally, the authors consider a ``tree order'' setting in which the null
		hypothesis is that all treatments have the same effects, and the
		alternative is that all treatments $2\leq i\leq K$ have an effect
		greater than treatment $1$. The Nash equilibrium of the two-player game
		is computed.
\end{itemize}

\paragraph{Overall impression.} The problem considered in this paper is natural
and interesting but I found the results to be underwhelming. Specifically:
\begin{itemize}
	\item the fact that the balanced design is optimal was already known for
		some criteria (e.g. A-optimality and, by a previous of the authors, the
		maxi-min criterion). As revealed by the proofs, all of these results
		(including the new criteria considered in this work) express in
		slightly different ways that the underlying ANOVA setting is symmetric.
	\item I do not completely buy the game theoretic formulation. There is
		nothing inherently strategic in the setting considered.  The fact that
		any maxi-min or mini-max problem can be \emph{interpreted} as
		a two-player game is always true and it does not seem that the present
		paper is saying more than that.
	\item I am not convinced by the value of arbitrarily restricting the action
		space of the max-player (the experiment designer) at the beginning of
		Section 4 and of the resulting Theorem 4.1. Would it make more sense to
		directly consider the general case where the max-player can choose
		\emph{any} design, and in particular, where the balanced design is part
		of the action space.  In other words, the story leading to Theorem 4.3
		seems unnecessary to me and I think this theorem could have been stated
		first.
\end{itemize}

\paragraph{Minor comments.}
\begin{itemize}
	\item  In the definition of $\mathcal{M}_\delta$ (equation (7)), it would
		be good to have a remark about how $\delta$ is chosen. For examples,
		what are implications of choosing one value vs.\ another value, how to
		choose it in practice, etc.
	\item In section 4, I think it would be better to define the value of game
		outside of Theorem 4.2. Defining it formally, explain that it is only
		defined when $\min\max\dots = \max\min\dots$  and that it requires the
		action space to be convex.
	\item related to the previous point, the use of the word \emph{value} in
		the proofs is inconsistent. It seems to be sometimes used to refer the
		value of an arbitrary pair of strategies, whereas in game theory, it
		usually only refers to the $\min\max$ value.
	\item paragraph above Theorem 4.3: the phrasing ``Since any $\xi$ can be
		written as mixture of other elements\dots'' is a bit vague. It would be
		better to simply say: ``Since the action space $\Xi$ is convex''.
		Relatedly, when introducing mixed strategies, I would emphasize the
		importance of having a convex strategy space, which is the key property
		needed to guarantee the existence of Nash equilibria and the reason to
		consider mixed strategies in the first place, when the action space is
		finite.
\end{itemize}

\paragraph{Conclusion.} The paper is well-written and constitutes a clean and
short exposition of simple results. 

\end{document}