Skip to content

Commit 9cf6a68

Browse files
styvonstyvon
authored
add vignettes
1 parent b7449e2 commit 9cf6a68

File tree

2 files changed

+176
-0
lines changed

2 files changed

+176
-0
lines changed

‎vignettes/cpcg-intro.Rmd

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
---
2+
title: "cpcg-intro"
3+
author: "Yongwen Zhuang"
4+
date: "`r Sys.Date()`"
5+
output: rmarkdown::html_vignette
6+
vignette: >
7+
%\VignetteIndexEntry{cpcg-intro}
8+
%\VignetteEngine{knitr::rmarkdown}
9+
%\VignetteEncoding{UTF-8}
10+
---
11+
12+
```{r setup, include = FALSE}
13+
knitr::opts_chunk$set(
14+
collapse = TRUE,
15+
comment = "#>"
16+
)
17+
```
18+
19+
Functions in this package serve the purpose of solving for $\boldsymbol{x}$ in $\boldsymbol{Ax=b}$, where $\boldsymbol{A}$ is a $n \times n$ symmetric and positive definite matrix, $\boldsymbol{b}$ is a $n \times 1$ column vector.
20+
21+
To improve scalability of conjugate gradient methods for larger matrices, the C++ Armadillo templated linear algebra library is used for the implementation. The package also provides flexibility to have user-specified preconditioner options to cater for different optimization needs.
22+
23+
This vignette will walk through some simple examples for using main functions in the package.
24+
25+
## 1. `cgsolve()`: Conjugate gradient method
26+
27+
The idea of conjugate gradient method is to find a set of mutually conjugate directions for the unconstrained problem $$\arg \min_x f(x)$$ where $f(x) = 0.5 y^T \Sigma y - yx + z$ and $z$ is a constant. The problem is equivalent to solving $\Sigma x = y$.
28+
29+
This function implements an iterative procedure to reduce the number of matrix-vector multiplications. The conjugate gradient method improves memory efficiency and computational complexity, especially when $\Sigma$ is relatively sparse.
30+
31+
**Example**: Solve $Ax = b$ where $A = \begin{bmatrix}
32+
4 & 1 \\
33+
1 & 3
34+
\end{bmatrix}$, $b = \begin{bmatrix}
35+
1 \\
36+
2
37+
\end{bmatrix}$.
38+
39+
```{r, eval=FALSE}
40+
test_A <- matrix(c(4,1,1,3), ncol = 2)
41+
test_b <- matrix(1:2, ncol = 1)
42+
43+
cgsolve(test_A, test_b, 1e-6, 1000)
44+
```
45+
46+
## 2. `pcgsolve()`: Preconditioned conjugate gradient method
47+
48+
When the condition number for $\Sigma$ is large, the conjugate gradient (CG) method may fail to converge in a reasonable number of iterations. The Preconditioned Conjugate Gradient (PCG) Method applies a precondition matrix $C$ and approaches the problem by solving: $$C^{-1} \Sigma x = C^{-1} y$$ where the symmetric and positive-definite matrix $C$ approximates $\Sigma$ and $C^{-1} \Sigma$ improves the condition number of $\Sigma$.
49+
50+
Choices for the preconditioner include: Jacobi preconditioning (`Jacobi`), symmetric successive over-relaxation (`SSOR`), and incomplete Cholesky factorization (`ICC`).
51+
**Example revisited**: Now we solve the same problem using incomplete Cholesky factorization of $A$ as preconditioner.
52+
53+
```{r, eval=FALSE}
54+
test_A <- matrix(c(4,1,1,3), ncol = 2)
55+
test_b <- matrix(1:2, ncol = 1)
56+
57+
pcgsolve(test_A, test_b, "ICC")
58+
```
59+
60+
61+
*Check [Github repo](https://github.com/styvon/cPCG/) and [cPCG: Efficient and Customized Preconditioned Conjugate Gradient Method](https://github.com/styvon/cPCG/blob/master/docs/article_cPCG.pdf) for more information.*

‎vignettes/cpcg-intro.html

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
<!DOCTYPE html>
2+
3+
<html xmlns="http://www.w3.org/1999/xhtml">
4+
5+
<head>
6+
7+
<meta charset="utf-8" />
8+
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
9+
<meta name="generator" content="pandoc" />
10+
11+
<meta name="viewport" content="width=device-width, initial-scale=1">
12+
13+
<meta name="author" content="Yongwen Zhuang" />
14+
15+
<meta name="date" content="2018-11-26" />
16+
17+
<title>cpcg-intro</title>
18+
19+
20+
21+
<style type="text/css">code{white-space: pre;}</style>
22+
<style type="text/css">
23+
div.sourceCode { overflow-x: auto; }
24+
table.sourceCode, tr.sourceCode, td.lineNumbers, td.sourceCode {
25+
margin: 0; padding: 0; vertical-align: baseline; border: none; }
26+
table.sourceCode { width: 100%; line-height: 100%; }
27+
td.lineNumbers { text-align: right; padding-right: 4px; padding-left: 4px; color: #aaaaaa; border-right: 1px solid #aaaaaa; }
28+
td.sourceCode { padding-left: 5px; }
29+
code > span.kw { color: #007020; font-weight: bold; } /* Keyword */
30+
code > span.dt { color: #902000; } /* DataType */
31+
code > span.dv { color: #40a070; } /* DecVal */
32+
code > span.bn { color: #40a070; } /* BaseN */
33+
code > span.fl { color: #40a070; } /* Float */
34+
code > span.ch { color: #4070a0; } /* Char */
35+
code > span.st { color: #4070a0; } /* String */
36+
code > span.co { color: #60a0b0; font-style: italic; } /* Comment */
37+
code > span.ot { color: #007020; } /* Other */
38+
code > span.al { color: #ff0000; font-weight: bold; } /* Alert */
39+
code > span.fu { color: #06287e; } /* Function */
40+
code > span.er { color: #ff0000; font-weight: bold; } /* Error */
41+
code > span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */
42+
code > span.cn { color: #880000; } /* Constant */
43+
code > span.sc { color: #4070a0; } /* SpecialChar */
44+
code > span.vs { color: #4070a0; } /* VerbatimString */
45+
code > span.ss { color: #bb6688; } /* SpecialString */
46+
code > span.im { } /* Import */
47+
code > span.va { color: #19177c; } /* Variable */
48+
code > span.cf { color: #007020; font-weight: bold; } /* ControlFlow */
49+
code > span.op { color: #666666; } /* Operator */
50+
code > span.bu { } /* BuiltIn */
51+
code > span.ex { } /* Extension */
52+
code > span.pp { color: #bc7a00; } /* Preprocessor */
53+
code > span.at { color: #7d9029; } /* Attribute */
54+
code > span.do { color: #ba2121; font-style: italic; } /* Documentation */
55+
code > span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */
56+
code > span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */
57+
code > span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */
58+
</style>
59+
60+
61+
62+
<link href="data:text/css;charset=utf-8,body%20%7B%0Abackground%2Dcolor%3A%20%23fff%3B%0Amargin%3A%201em%20auto%3B%0Amax%2Dwidth%3A%20700px%3B%0Aoverflow%3A%20visible%3B%0Apadding%2Dleft%3A%202em%3B%0Apadding%2Dright%3A%202em%3B%0Afont%2Dfamily%3A%20%22Open%20Sans%22%2C%20%22Helvetica%20Neue%22%2C%20Helvetica%2C%20Arial%2C%20sans%2Dserif%3B%0Afont%2Dsize%3A%2014px%3B%0Aline%2Dheight%3A%201%2E35%3B%0A%7D%0A%23header%20%7B%0Atext%2Dalign%3A%20center%3B%0A%7D%0A%23TOC%20%7B%0Aclear%3A%20both%3B%0Amargin%3A%200%200%2010px%2010px%3B%0Apadding%3A%204px%3B%0Awidth%3A%20400px%3B%0Aborder%3A%201px%20solid%20%23CCCCCC%3B%0Aborder%2Dradius%3A%205px%3B%0Abackground%2Dcolor%3A%20%23f6f6f6%3B%0Afont%2Dsize%3A%2013px%3B%0Aline%2Dheight%3A%201%2E3%3B%0A%7D%0A%23TOC%20%2Etoctitle%20%7B%0Afont%2Dweight%3A%20bold%3B%0Afont%2Dsize%3A%2015px%3B%0Amargin%2Dleft%3A%205px%3B%0A%7D%0A%23TOC%20ul%20%7B%0Apadding%2Dleft%3A%2040px%3B%0Amargin%2Dleft%3A%20%2D1%2E5em%3B%0Amargin%2Dtop%3A%205px%3B%0Amargin%2Dbottom%3A%205px%3B%0A%7D%0A%23TOC%20ul%20ul%20%7B%0Amargin%2Dleft%3A%20%2D2em%3B%0A%7D%0A%23TOC%20li%20%7B%0Aline%2Dheight%3A%2016px%3B%0A%7D%0Atable%20%7B%0Amargin%3A%201em%20auto%3B%0Aborder%2Dwidth%3A%201px%3B%0Aborder%2Dcolor%3A%20%23DDDDDD%3B%0Aborder%2Dstyle%3A%20outset%3B%0Aborder%2Dcollapse%3A%20collapse%3B%0A%7D%0Atable%20th%20%7B%0Aborder%2Dwidth%3A%202px%3B%0Apadding%3A%205px%3B%0Aborder%2Dstyle%3A%20inset%3B%0A%7D%0Atable%20td%20%7B%0Aborder%2Dwidth%3A%201px%3B%0Aborder%2Dstyle%3A%20inset%3B%0Aline%2Dheight%3A%2018px%3B%0Apadding%3A%205px%205px%3B%0A%7D%0Atable%2C%20table%20th%2C%20table%20td%20%7B%0Aborder%2Dleft%2Dstyle%3A%20none%3B%0Aborder%2Dright%2Dstyle%3A%20none%3B%0A%7D%0Atable%20thead%2C%20table%20tr%2Eeven%20%7B%0Abackground%2Dcolor%3A%20%23f7f7f7%3B%0A%7D%0Ap%20%7B%0Amargin%3A%200%2E5em%200%3B%0A%7D%0Ablockquote%20%7B%0Abackground%2Dcolor%3A%20%23f6f6f6%3B%0Apadding%3A%200%2E25em%200%2E75em%3B%0A%7D%0Ahr%20%7B%0Aborder%2Dstyle%3A%20solid%3B%0Aborder%3A%20none%3B%0Aborder%2Dtop%3A%201px%20solid%20%23777%3B%0Amargin%3A%2028px%200%3B%0A%7D%0Adl%20%7B%0Amargin%2Dleft%3A%200%3B%0A%7D%0Adl%20dd%20%7B%0Amargin%2Dbottom%3A%2013px%3B%0Amargin%2Dleft%3A%2013px%3B%0A%7D%0Adl%20dt%20%7B%0Afont%2Dweight%3A%20bold%3B%0A%7D%0Aul%20%7B%0Amargin%2Dtop%3A%200%3B%0A%7D%0Aul%20li%20%7B%0Alist%2Dstyle%3A%20circle%20outside%3B%0A%7D%0Aul%20ul%20%7B%0Amargin%2Dbottom%3A%200%3B%0A%7D%0Apre%2C%20code%20%7B%0Abackground%2Dcolor%3A%20%23f7f7f7%3B%0Aborder%2Dradius%3A%203px%3B%0Acolor%3A%20%23333%3B%0Awhite%2Dspace%3A%20pre%2Dwrap%3B%20%0A%7D%0Apre%20%7B%0Aborder%2Dradius%3A%203px%3B%0Amargin%3A%205px%200px%2010px%200px%3B%0Apadding%3A%2010px%3B%0A%7D%0Apre%3Anot%28%5Bclass%5D%29%20%7B%0Abackground%2Dcolor%3A%20%23f7f7f7%3B%0A%7D%0Acode%20%7B%0Afont%2Dfamily%3A%20Consolas%2C%20Monaco%2C%20%27Courier%20New%27%2C%20monospace%3B%0Afont%2Dsize%3A%2085%25%3B%0A%7D%0Ap%20%3E%20code%2C%20li%20%3E%20code%20%7B%0Apadding%3A%202px%200px%3B%0A%7D%0Adiv%2Efigure%20%7B%0Atext%2Dalign%3A%20center%3B%0A%7D%0Aimg%20%7B%0Abackground%2Dcolor%3A%20%23FFFFFF%3B%0Apadding%3A%202px%3B%0Aborder%3A%201px%20solid%20%23DDDDDD%3B%0Aborder%2Dradius%3A%203px%3B%0Aborder%3A%201px%20solid%20%23CCCCCC%3B%0Amargin%3A%200%205px%3B%0A%7D%0Ah1%20%7B%0Amargin%2Dtop%3A%200%3B%0Afont%2Dsize%3A%2035px%3B%0Aline%2Dheight%3A%2040px%3B%0A%7D%0Ah2%20%7B%0Aborder%2Dbottom%3A%204px%20solid%20%23f7f7f7%3B%0Apadding%2Dtop%3A%2010px%3B%0Apadding%2Dbottom%3A%202px%3B%0Afont%2Dsize%3A%20145%25%3B%0A%7D%0Ah3%20%7B%0Aborder%2Dbottom%3A%202px%20solid%20%23f7f7f7%3B%0Apadding%2Dtop%3A%2010px%3B%0Afont%2Dsize%3A%20120%25%3B%0A%7D%0Ah4%20%7B%0Aborder%2Dbottom%3A%201px%20solid%20%23f7f7f7%3B%0Amargin%2Dleft%3A%208px%3B%0Afont%2Dsize%3A%20105%25%3B%0A%7D%0Ah5%2C%20h6%20%7B%0Aborder%2Dbottom%3A%201px%20solid%20%23ccc%3B%0Afont%2Dsize%3A%20105%25%3B%0A%7D%0Aa%20%7B%0Acolor%3A%20%230033dd%3B%0Atext%2Ddecoration%3A%20none%3B%0A%7D%0Aa%3Ahover%20%7B%0Acolor%3A%20%236666ff%3B%20%7D%0Aa%3Avisited%20%7B%0Acolor%3A%20%23800080%3B%20%7D%0Aa%3Avisited%3Ahover%20%7B%0Acolor%3A%20%23BB00BB%3B%20%7D%0Aa%5Bhref%5E%3D%22http%3A%22%5D%20%7B%0Atext%2Ddecoration%3A%20underline%3B%20%7D%0Aa%5Bhref%5E%3D%22https%3A%22%5D%20%7B%0Atext%2Ddecoration%3A%20underline%3B%20%7D%0A%0Acode%20%3E%20span%2Ekw%20%7B%20color%3A%20%23555%3B%20font%2Dweight%3A%20bold%3B%20%7D%20%0Acode%20%3E%20span%2Edt%20%7B%20color%3A%20%23902000%3B%20%7D%20%0Acode%20%3E%20span%2Edv%20%7B%20color%3A%20%2340a070%3B%20%7D%20%0Acode%20%3E%20span%2Ebn%20%7B%20color%3A%20%23d14%3B%20%7D%20%0Acode%20%3E%20span%2Efl%20%7B%20color%3A%20%23d14%3B%20%7D%20%0Acode%20%3E%20span%2Ech%20%7B%20color%3A%20%23d14%3B%20%7D%20%0Acode%20%3E%20span%2Est%20%7B%20color%3A%20%23d14%3B%20%7D%20%0Acode%20%3E%20span%2Eco%20%7B%20color%3A%20%23888888%3B%20font%2Dstyle%3A%20italic%3B%20%7D%20%0Acode%20%3E%20span%2Eot%20%7B%20color%3A%20%23007020%3B%20%7D%20%0Acode%20%3E%20span%2Eal%20%7B%20color%3A%20%23ff0000%3B%20font%2Dweight%3A%20bold%3B%20%7D%20%0Acode%20%3E%20span%2Efu%20%7B%20color%3A%20%23900%3B%20font%2Dweight%3A%20bold%3B%20%7D%20%20code%20%3E%20span%2Eer%20%7B%20color%3A%20%23a61717%3B%20background%2Dcolor%3A%20%23e3d2d2%3B%20%7D%20%0A" rel="stylesheet" type="text/css" />
63+
64+
</head>
65+
66+
<body>
67+
68+
69+
70+
71+
<h1 class="title toc-ignore">cpcg-intro</h1>
72+
<h4 class="author"><em>Yongwen Zhuang</em></h4>
73+
<h4 class="date"><em>2018-11-26</em></h4>
74+
75+
76+
77+
<p>Functions in this package serve the purpose of solving for <span class="math inline">\(\boldsymbol{x}\)</span> in <span class="math inline">\(\boldsymbol{Ax=b}\)</span>, where <span class="math inline">\(\boldsymbol{A}\)</span> is a <span class="math inline">\(n \times n\)</span> symmetric and positive definite matrix, <span class="math inline">\(\boldsymbol{b}\)</span> is a <span class="math inline">\(n \times 1\)</span> column vector.</p>
78+
<p>To improve scalability of conjugate gradient methods for larger matrices, the C++ Armadillo templated linear algebra library is used for the implementation. The package also provides flexibility to have user-specified preconditioner options to cater for different optimization needs.</p>
79+
<p>This vignette will walk through some simple examples for using main functions in the package.</p>
80+
<div id="cgsolve-conjugate-gradient-method" class="section level2">
81+
<h2>1. <code>cgsolve()</code>: Conjugate gradient method</h2>
82+
<p>The idea of conjugate gradient method is to find a set of mutually conjugate directions for the unconstrained problem <span class="math display">\[\arg \min_x f(x)\]</span> where <span class="math inline">\(f(x) = 0.5 y^T \Sigma y - yx + z\)</span> and <span class="math inline">\(z\)</span> is a constant. The problem is equivalent to solving <span class="math inline">\(\Sigma x = y\)</span>.</p>
83+
<p>This function implements an iterative procedure to reduce the number of matrix-vector multiplications. The conjugate gradient method improves memory efficiency and computational complexity, especially when <span class="math inline">\(\Sigma\)</span> is relatively sparse.</p>
84+
<p><strong>Example</strong>: Solve <span class="math inline">\(Ax = b\)</span> where <span class="math inline">\(A = \begin{bmatrix} 4 &amp; 1 \\ 1 &amp; 3 \end{bmatrix}\)</span>, <span class="math inline">\(b = \begin{bmatrix} 1 \\ 2 \end{bmatrix}\)</span>.</p>
85+
<div class="sourceCode"><pre class="sourceCode r"><code class="sourceCode r">test_A &lt;-<span class="st"> </span><span class="kw">matrix</span>(<span class="kw">c</span>(<span class="dv">4</span>,<span class="dv">1</span>,<span class="dv">1</span>,<span class="dv">3</span>), <span class="dt">ncol =</span> <span class="dv">2</span>)
86+
test_b &lt;-<span class="st"> </span><span class="kw">matrix</span>(<span class="dv">1</span><span class="op">:</span><span class="dv">2</span>, <span class="dt">ncol =</span> <span class="dv">1</span>)
87+
88+
<span class="kw">cgsolve</span>(test_A, test_b, <span class="fl">1e-6</span>, <span class="dv">1000</span>)</code></pre></div>
89+
</div>
90+
<div id="pcgsolve-preconditioned-conjugate-gradient-method" class="section level2">
91+
<h2>2. <code>pcgsolve()</code>: Preconditioned conjugate gradient method</h2>
92+
<p>When the condition number for <span class="math inline">\(\Sigma\)</span> is large, the conjugate gradient (CG) method may fail to converge in a reasonable number of iterations. The Preconditioned Conjugate Gradient (PCG) Method applies a precondition matrix <span class="math inline">\(C\)</span> and approaches the problem by solving: <span class="math display">\[C^{-1} \Sigma x = C^{-1} y\]</span> where the symmetric and positive-definite matrix <span class="math inline">\(C\)</span> approximates <span class="math inline">\(\Sigma\)</span> and <span class="math inline">\(C^{-1} \Sigma\)</span> improves the condition number of <span class="math inline">\(\Sigma\)</span>.</p>
93+
<p>Choices for the preconditioner include: Jacobi preconditioning (<code>Jacobi</code>), symmetric successive over-relaxation (<code>SSOR</code>), and incomplete Cholesky factorization (<code>ICC</code>).<br />
94+
<strong>Example revisited</strong>: Now we solve the same problem using incomplete Cholesky factorization of <span class="math inline">\(A\)</span> as preconditioner.</p>
95+
<div class="sourceCode"><pre class="sourceCode r"><code class="sourceCode r">test_A &lt;-<span class="st"> </span><span class="kw">matrix</span>(<span class="kw">c</span>(<span class="dv">4</span>,<span class="dv">1</span>,<span class="dv">1</span>,<span class="dv">3</span>), <span class="dt">ncol =</span> <span class="dv">2</span>)
96+
test_b &lt;-<span class="st"> </span><span class="kw">matrix</span>(<span class="dv">1</span><span class="op">:</span><span class="dv">2</span>, <span class="dt">ncol =</span> <span class="dv">1</span>)
97+
98+
<span class="kw">pcgsolve</span>(test_A, test_b, <span class="st">&quot;ICC&quot;</span>)</code></pre></div>
99+
<p><em>Check <a href="https://github.com/styvon/cPCG/">Github repo</a> and <a href="https://github.com/styvon/cPCG/blob/master/docs/article_cPCG.pdf">cPCG: Efficient and Customized Preconditioned Conjugate Gradient Method</a> for more information.</em></p>
100+
</div>
101+
102+
103+
104+
<!-- dynamically load mathjax for compatibility with self-contained -->
105+
<script>
106+
(function () {
107+
var script = document.createElement("script");
108+
script.type = "text/javascript";
109+
script.src = "https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML";
110+
document.getElementsByTagName("head")[0].appendChild(script);
111+
})();
112+
</script>
113+
114+
</body>
115+
</html>

0 commit comments

Comments
 (0)