blob: c021c3165205e251e36587b9acc9bd406a53c351 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
|
\documentclass[11pt]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage[hmargin=1.2in, vmargin=1.2in]{geometry}
\usepackage{amsmath,amsfonts}
\title{\vspace{-2em}\large Review of \emph{A Multi-Agent Reinforcement Learning
Framework\\ for Off-Policy Evaluation in Two-Sided Markets}}
\author{Submission \textsf{2103--005} to the \emph{Annals of Applied Statistics},
1\textsuperscript{st} revision}
\begin{document}
\maketitle
I would like to thank the authors for carefully addressing all my comments in
their response and revision. In particular, I noted significant improvements
regarding the framing and contributions of the paper in Section 1, and
clarifications regarding the setup and assumptions in Section 2. I also
appreciate the improved discussion on the mean-field approximation and how to
test it empirically using a conditional independence test. I only have two
minor comments:
\begin{itemize}
\item I believe there a few inaccuracies in the derivation in Appendix C:
\begin{itemize}
\item in the first displayed expression for $Q_i^{\bf\pi}({\bf a},{\bf
s})$, the parenthesis after $m_i^a({\bf A_t})$ should instead be
placed after $\tilde S_{i,t}$. Same remark about the inlined
expression on the next line.
\item the sequence of equalities on page 25 should be stated for an
arbitrary square-integrable function $h$ instead of $\bar r_i$
to be consistent with the induction hypothesis.
\item in the same sequence of equalities, the last equality should
have $m_i^a(\pi({\bf S}_{j}))$ instead of $m_i^a(\pi({\bf
S}_{j+1}))$ in the “conditioning” part (after the ‘$|$’
character).
\end{itemize}
\item In Appendix E in the supplementary material, I suggest adding plots
similar to the ones in Fig. 4 of the main file, but only showing the
two curves corresponding to the proposed method and the QV baseline.
This is because these two curves are so close together in the current
Fig.\ 4 that it is hard to tell them apart due to the scale imposed by the
other curves. My hope is that restricting to these two curves only in
the appendix will allow for a better scale on the $y$-axis that will
make them easier to distinguish and provide visual support for the
$t$-test reported in Table 1 and 2.
\end{itemize}
I recommend acceptance after the above minor points are addressed.
\vspace{1em}
\paragraph{Additional typos.}
\begin{itemize}
\item page 4, second paragraph: \emph{Event through} $\to$ \emph{Even though}
\item the running title at the top of each page does not match the new
paper title.
\end{itemize}
%\bibliographystyle{plain}
%\bibliography{main}
\end{document}
|
