Skip to content
Daniel edited this page May 19, 2017 · 2 revisions

Consistency Checks for Multipole Fits

Mock Dataset Tests

Generate a toy mock dataset from a distortion-free theory + noise sampled from a diagonal DR10 icov from Timothée. Binning is (rP,rT) with [0:200]*40 (5 Mpc/h bins). (The failed to estimate full covariance matrix error is because we only generate one MC sample, and is safe to ignore.)

./baofit -i ../config/comoving.ini --data rTPFullDiag --load-icov --output-prefix mock_ --rmin 0 --rmax 500 --toymc-samples 1 --toymc-save --no-initial-fit --ndump 0
mv mock_toymcsave.data mock.data
cp rTPFullDiag.icov mock.icov

Now do a cosmo fit to this toy mock with only beta, alpha-par and alpha-perp floating, and 40 <= r <= 180 Mpc/h:

./baofit -i ../config/comoving.ini --data mock --load-icov --dist-add "" --dist-mul "" --output-prefix mock_fit_

Fit results: chiSquare / dof = 953.019 / (963-3), prob = 0.557438, log(det(Covariance)) = -21069

FMIN Value =   476.509617 at:
               beta =     1.415525 +/-     0.068162 $    1.42 \pm 0.07 $
 BAO alpha-parallel =     1.002008 +/-     0.018076 $  1.002 \pm 0.018 $
     BAO alpha-perp =     0.972253 +/-     0.028145 $  0.972 \pm 0.028 $

FMIN Errors & Correlations =
              beta :     0.068162
BAO alpha-parallel :    -0.015233     0.018076
    BAO alpha-perp :     0.027790    -0.413302     0.028145

Next, fit the same toy mock to find its best fit multipoles at r = 40,45,50,...,180 (29 points). First, we fit all multipoles (ell = 0,2,4) for a total of 3*29 = 87 params:

./baofit -i ../config/comoving_xi.ini --data mock --load-icov --output-prefix mock_ell024_ \
--xi-points 40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180 

Fit results: chiSquare / dof = 847.708 / (963-87), prob = 0.74773, log(det(Covariance)) = -21069

     Xi y-0-0 =     6.139316 +/-     0.697306 $      6.1 \pm 0.7 $
     Xi y-0-1 =     3.835945 +/-     0.418932 $      3.8 \pm 0.4 $
     Xi y-0-2 =     3.794558 +/-     0.391574 $      3.8 \pm 0.4 $
...

For comparison, we also fit just the mono+quad (ell = 0,2) using model-config to fix the hexadecapole parameters to zero:

./baofit -i ../config/comoving_xi.ini --data mock --load-icov --output-prefix mock_ell02_ --model-config="fix[Xi y-4-*]=0" \
--xi-points 40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180 

Fit results: chiSquare / dof = 960.433 / (963-58), prob = 0.0980324, log(det(Covariance)) = -21069

FMIN Value =   480.216266 at:
         beta =     1.400000
(1+beta)*bias =    -0.336000
   gamma-bias =     3.800000
   gamma-beta =     0.000000
     Xi y-0-0 =     6.419652 +/-     0.685418 $      6.4 \pm 0.7 $
     Xi y-0-1 =     3.718524 +/-     0.409617 $      3.7 \pm 0.4 $
     Xi y-0-2 =     3.874894 +/-     0.384302 $      3.9 \pm 0.4 $
...

Both of these fits output their parameters in a format that can be used as input to a multipole fit, and print a message like this:

Use the following config options to refit this data using the fitted multipoles:

data = mock_ell02_multipoles
data-format = comoving-multipole
axis1-bins = {40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180}
axis2-bins = {0,2,4}
axis3-bins = {2.25}

Next, fit using the config options given above (need to comment out load-icov in comoving-xi.ini first, and binning options must be quoted on the command line). Results are essentially identical to those above, as expected, although the fit now has npar = nbins so the chisq is meaningless. First, the ell=0,2,4 multipoles:

./baofit -i ../config/comoving_xi.ini --data mock_ell024_multipoles --data-format comoving-multipole \
--xi-points 40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180 \
--axis1-bins "{40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180}" --axis2-bins "{0,2,4}" --axis3-bins "{2.25}" \
--lmin 0 --lmax 4 --output-prefix mock_refit_ell024_

Fit is ill-conditioned with nbins = 87, npar = 87, chiSquare = 3.20323e-06

     Xi y-0-0 =     6.139555 +/-     0.697306 $      6.1 \pm 0.7 $
     Xi y-0-1 =     3.835892 +/-     0.418932 $      3.8 \pm 0.4 $
     Xi y-0-2 =     3.794711 +/-     0.391574 $      3.8 \pm 0.4 $
...

Second, the ell = 0,2 multipoles (where we need to fix the ell=4 parameters, as above):

./baofit -i ../config/comoving_xi.ini --data mock_ell02_multipoles --data-format comoving-multipole \
--xi-points 40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180 \
--axis1-bins "{40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180}" --axis2-bins "{0,2,4}" --axis3-bins "{2.25}" \
--lmin 0 --lmax 2 --model-config="fix[Xi y-4-*]=0" --output-prefix mock_refit_ell02_

Fit is ill-conditioned with nbins = 58, npar = 58, chiSquare = 1.1299e-06

     Xi y-0-0 =     6.419722 +/-     0.685418 $      6.4 \pm 0.7 $
     Xi y-0-1 =     3.718507 +/-     0.409617 $      3.7 \pm 0.4 $
     Xi y-0-2 =     3.874931 +/-     0.384302 $      3.9 \pm 0.4 $
...

Finally, do the original cosmology fit to the multipoles in several different ways. First, fit ell = 0,2,4 to use the full multipole information:

./baofit -i ../config/comoving.ini --data mock_ell024_multipoles --data-format comoving-multipole \
--axis1-bins "{40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180}" --axis2-bins "{0,2,4}" --axis3-bins "{2.25}" \
--lmin 0 --lmax 4 --dist-add "" --dist-mul "" --output-prefix mock_ell024_fit024_

Fit results: chiSquare / dof = 105.75 / (87-3), prob = 0.0545525, log(det(Covariance)) = -2073.32

FMIN Value =    52.874966 at:
               beta =     1.414014 +/-     0.068171 $    1.41 \pm 0.07 $
 BAO alpha-parallel =     1.001841 +/-     0.018064 $  1.002 \pm 0.018 $
     BAO alpha-perp =     0.970763 +/-     0.028428 $  0.971 \pm 0.028 $

FMIN Errors & Correlations =
              beta :     0.068171
BAO alpha-parallel :    -0.013024     0.018064
    BAO alpha-perp :     0.025535    -0.405439     0.028428

Next, only use the ell = 0,2 subset of the full ell = 0,2,4 dataset (and covariance) in the fit (by changing lmax):

./baofit -i ../config/comoving.ini --data mock_ell024_multipoles --data-format comoving-multipole \
--axis1-bins "{40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180}" --axis2-bins "{0,2,4}" --axis3-bins "{2.25}" \
--lmin 0 --lmax 2 --dist-add "" --dist-mul "" --output-prefix mock_ell024_fit02_

Fit results: chiSquare / dof = 55.859 / (58-3), prob = 0.44235, log(det(Covariance)) = -1409.34

               beta =     1.445337 +/-     0.076822 $    1.45 \pm 0.08 $
 BAO alpha-parallel =     1.015529 +/-     0.021743 $  1.016 \pm 0.022 $
     BAO alpha-perp =     0.951949 +/-     0.031385 $  0.952 \pm 0.031 $

FMIN Errors & Correlations =
              beta :     0.076822
BAO alpha-parallel :    -0.016388     0.021743
    BAO alpha-perp :     0.023216    -0.563814     0.031385

Next, fit the ell = 0,2 dataset:

./baofit -i ../config/comoving.ini --data mock_ell02_multipoles --data-format comoving-multipole \
--axis1-bins "{40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180}" --axis2-bins "{0,2,4}" --axis3-bins "{2.25}" \
--lmin 0 --lmax 2 --dist-add "" --dist-mul "" --output-prefix mock_ell02_fit02_

Fit results: chiSquare / dof = 62.8448 / (58-3), prob = 0.218285, log(det(Covariance)) = -1409.72

               beta =     1.427838 +/-     0.075144 $    1.43 \pm 0.08 $
 BAO alpha-parallel =     1.015978 +/-     0.021666 $  1.016 \pm 0.022 $
     BAO alpha-perp =     0.951570 +/-     0.031220 $  0.952 \pm 0.031 $

FMIN Errors & Correlations =
              beta :     0.075144
BAO alpha-parallel :    -0.016458     0.021666
    BAO alpha-perp :     0.023123    -0.563685     0.031220

2D vs Multipole Sensitivity

Fit Dataset Binning Nbins Cuts Nbins(cut) alpha(par) alpha(perp)
1 0 <= rP,rT <= 200 40^2^ 40 <= r <= 180 963 1.002 +- 0.018 0.972 +- 0.028
2 40 <= r <= 180, ell = 0,2,4 3*29 none 87 1.002 +- 0.018 0.971 +- 0.028
3 40 <= r <= 180, ell = 0,2,4 3*29 ell = 0,2 58 1.016 +- 0.022 0.952 +- 0.031
4 40 <= r <= 180, ell = 0,2 2*29 none 58 1.016 +- 0.022 0.952 +- 0.031

Add the options --model-config "binning[BAO alpha-p*]={0.9:1.1}*41" --parameter-scan --ndump 0 to generate the chisq scans for fits 1 and 4, marginalized over beta:

./baofit -i ../config/comoving.ini --data mock --load-icov --dist-add "" --dist-mul "" --output-prefix mock_fit_ --model-config "binning[BAO alpha-p*]={0.8:1.2}*81" --parameter-scan --ndump 0
./baofit -i ../config/comoving.ini --data mock_ell02_multipoles --data-format comoving-multipole \
--axis1-bins "{40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180}" --axis2-bins "{0,2,4}" --axis3-bins "{2.25}" \
--lmin 0 --lmax 2 --dist-add "" --dist-mul "" --output-prefix mock_ell02_fit02_ --model-config "binning[BAO alpha-p*]={0.8:1.2}*81" --parameter-scan --ndump 0

Use Mathematica to load chi-square scans:

Needs["DeepZot`BaoFitTools`"]
loadFitAnalysis[fit1scan, "mock_fit_scan", path -> "/Users/david/Cosmo/LyAlpha/code/baofit/build"]
loadFitAnalysis[fit4scan, "mock_ell02_fit02_scan", path -> "/Users/david/Cosmo/LyAlpha/code/baofit/build"]

and make plots at 68% and 95% CL:

Needs["DeepZot`StatisticsTools`"]
With[{
  plist = {8, 7}, (* 8 = alpha(perp), 7 = alpha(par) *)   
  levels = gaussianChiSquareContourLevel[{0.68, 0.95}, 2]
},
Show[{
  createFrame[Plot, {0.84, 1.16}, {0.84, 1.16}, AxesOrigin -> {1, 1}, AxesStyle -> Dotted, AspectRatio -> 1, 
    FrameLabel -> {"\[Alpha](perp)", "\[Alpha](par)"}, LabelStyle -> Large, ImageSize -> 500],
  ListContourPlot[fitAnalysisSamples[fit1scan, parameters -> plist],
    ContourStyle -> Red, Contours -> levels, ContourShading -> None],
  ListContourPlot[fitAnalysisSamples[fit4scan, parameters -> plist],
    ContourStyle -> Directive[Dashed, Blue], Contours -> levels, ContourShading -> None]
}]]

{{attachment:chisq.png}}

Clone this wiki locally