1. The Art of Linear Algebra – Graphic Notes on “Linear Algebra for Everyone” – Kenji Hiranabe ∗ with the kindest help of Gilbert Strang translator: Kefang Liu ‡ † September 1, 2021/updated July 12, 2023 Abstract Gilbert Strang Linear Algebra for Everyone . 1 (Column-Row, CR) (Gaussian Elimination, LU ) - (Gram-Schmidt Orthogonalization, QR) (Eigenvalues and Diagonalization, QΛQ T ) (Singular Value Decomposition, , U ΣV T ). Kenji Hiranabe · ! , Kenji . . “ ” “ 1 ” . – Gilbert Strang Contents 1 4 2 2 2 2 3 2 3 4 4 4 5 4 6 6.1 6.2 6.3 6.4 6.5 A = CR . . A = LU . . A = QR . . S = QΛQ T . A = U ΣV T . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ∗ twitter: . . . . . . . . . . . . . . . . . . . . . . . . . @hiranabe, k-hiranabe@esm.co.jp, https://anagileway.com Institute of Technology, http://www-math.mit.edu/~gs/ ‡ twitter: @kfchliu, : 5717297833 1 “Linear Algebra for Everyone”: http://math.mit.edu/everyone/. † Massachusetts 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 7 8 8 9 10

2. 1 4 (m × n) , mn 1 , n ! ! ! . m # $ # $ # $ " ! % & ! % & ! % & ' ( ' ( ' ( 4"'5+,/6 3"(&'2*,- !"#$%&'(")*#+$,- ./+0 1"(&'2*,- 1",$.")*#+$,- ./+0 !"(&'2*,- Figure 1:  a 11 A =  a 21 a 31 ,   a 12 | a 22  =  a 1 a 32 | ∗ a 1 . T   ∗  | −a 1 − a 2  =  −a ∗ 2 −  −a ∗ 3 − | a ∗ 1 . , a T T A . 2 2 , , ). , Linear Algebra for Everyone , v1 ( 1 ) Mv1 ( , , , , (v1) . • 1.1 (p.2) Linear combination and dot products • 1.3 (p.25) Matrix of Rank One • 1.4 (p.29) Row way and column way ! !" ! !"#$%&"'()#$$*+(,-.&/ $ ! $ ! ! ! " # $ " & " ' $ " & $ ! ( "$ " ( #$ # $ # $ # # ), . ! !# ! " $ # Figure 2: , . 1 ! 01+2$3$41#&56 !" $ 57$1$,1#&56$#!" $ $ %&8$9:$+.5#;.&$'( ) 1&.$*( #;.$&.7(<#$% 57$1$&1+2$3$,1#&568 !"#$%&"'()# *! + "/$57$.6%&.77.'$17$! $ " 5+$ ,1#&56$<1+=(1=.$1+'$>5.<'7$1$+(,-.&8 (v1) ( $ % & "$ #$ % "% #% - (v1), (v2) (v2) 1 . 2 . (v2) ( 1)

3. 3 2 (Mv1) • 1.1 (p.3) Linear combinations • 1.3 (p.21) Matrices and Column Spaces ! !"# !"#$3%&40)*$:7 2+$.$128#.%$)&/528.*2&8$&,$*"#$ )&10/8$(#)*&%+$&,$ -; $ % * $ % ! !" # & ' * # * ! & , * " ' " ( ) ( ) +* ! ,%* " - $ % * ! !" # & ' * # +&* ! , '* " - " +(* ! , )* " - ( ) Figure 3: (Mv1). (vM1) (vM2) $ " , Ax . A , N(A). . % & $ # ' ( # +$ ! ,'$ " , )$ # - +&$ ! , ($ " , *$ # - ) * ! !"$ !" # $ ! , ! !"# !" # $ ! - (Mv1), (Mv2) (Mv2) C(A). Ax = 0 A " ! !"$ !"#$%&'$(#)*&%+$&,$ - .%#$/01*2312#4$56$.$(#)*&%$7 .84$5#)&/#$*"#$*"%##$4&*93%&40)*$#1#/#8*+$&,$:7; , . A $ " 8$&'6$7#*+'&$, ./$3)1+.%1.#($4:$ +"#$+6'$*'1)32$7#*+'&/$'5$ 8 02($4#*'3#$+"#$+6'$('+;%&'()*+$ #1#3#2+/$'5$,-9 " " % & $ # ' ( # $ ! % & , $ " ' ( , $ # ) * ) * Figure 4: !"#$%&'()*+$,- ./$0$1.2#0&$ *'34.20+.'2$'5$+"#$&'6$ 7#*+'&/$'5$ 89 - (vM1), (vM2) C(A T ). yA = 0 A A , T N(A ). : C(A) ( • 3.5 R n N(A) + C(A T ) ( ). (p.124) Dimensions of the Four Subspaces 3 ) R m N(A T ) +

4. " / ! "$! ! ! + " # ! " +,"- !"# % ' !"# % ' ,"-'()*!+ !"#$%&'()*!+ !""#$" !""#"# )+,)+&01!$#*, )+,)+&01!$#*, ! " ! ! &$##()*!+ .," # - "# % & .,"- !"# % ( ) ' #+./'&$##()*!+ $" % & !"# % * ) ' ! ! % . " 0 + " # . " 1 + " # ! " % + " 0 . " # 2 " 1 3 " # Figure 5: A = CR (6.1 r, 4 ). 4 “ ” • 1.4 “ ”. (p.35) Four ways to multiply AB = C • ! !! " !! # MN87O6898=84?6>8;<=8@6365<?6G7<5B;?6<A67<J6N8;?<7 3456;<9B=46N8;?<7D *' ! +"' " , *( ! +"( " , ! " ' ( ! ! # $ ' ( ) *#' ! +$' " , *#( ! +$( " , " " *%' ! +&' " , *%( ! +&( " , % & ! -. 3456-/ 37869:48376;<=>:43?:<4@6<A6;<9B=4@6<A6 CD ! " ' ( ! ! # $ ' ( ) - . / ) -. -/ " " % & !! % !! $ ! ! " ! ! FB9?:G9:;3?:<46-H :@6>7<I8465<J46?<636@B=6<A6734I6!6=3?7:;8@D KL86G7<5B;8567<J@637869:48376;<=>:43?:<4@6<A67<J@D 1 % # 1 % # E ! " ' ( ! ! # $ ' ( ) 1 % $ E ) 1 % $ E " " % & 1 % & E 1 % & Figure 6: ! # % ! ) # % ) 1 # 2 % # + 1 $ 2 % $ 0 !! 0 !" "0 "! "0 "" #0 !! #0 !" + $0 "! $0 "" %0 !! %0 !" &0 "! &0 "" - (MM1), (MM2), (MM3), (MM4) 5 , " 0 2 % 0 $ 0 !! 0 !" ) 1 # 1 $ # % 2 $ "! "" & " 0 !! 0 !" + $ 0 "! 0 "" ) & , 4

5. ! ! " # # # ! " # ! " !" ! !"#$%&'()*+,$(-+&.#+$'/.&+%0&+()+&.#+ 0(12-)*+(,+&.#+-%&$'34+5.'*+ #3"$#**'()+0%)+6#+*##)+%*+&.#+&.$##+ 1')#%$+0(-6')%&'()*+')+&.#+$'/.&+')+ ()#+,($-21%4 " # ! ! ! " # ! !"#$%&'()*+,$(-+&.#+1#,&+%0&+()+&.#+ $(7*+(,+&.#+-%&$'34+5.'*+#3"$#**'()+ 0%)+6#+*##)+%*+&.#+&.$##+1')#%$+ 0(-6')%&'()*+')+&.#+$'/.&+')+()#+ ,($-21%4 P1 (MM2) ( (Mv2) . P2 " " ! ! " " " " # " ! ! " " " # # ! ! " " " # Figure 7: P2 " " ! ! ! " # $%&'( $$ $%# # " ! # " !# " ! 1, 2 - (P1), (P1) (MM3) (vM2) . ! , P1 ( ), ! !$# !""#$%&'()(*%)'+&)#(,)-.%/(0.+,(-12(#20- 34)#23(2)41(.+76 !""#$%&'()(*%)'+&)#(,)-.%/(0.+,(-12(.%'1- 34)#23(2)41(4+#5,&6 % $ !" # $ ! $ " $ # % $ # % $ $ ! % % % % $ " % & $ # "& # % & % % % & (P1 ′ ) . (P2 ′ ) , . ! !" ' $' % $ ' $' ' ' % # % % ' ' % ' ' & % & ' ' & 1 ′ , 2 ′ - (P1 ′ ), (P2 ′ ) Figure 8: " " !"#$%&'(()*+%,'-)$%'+.(")*%/.,0#+'(#.+%.1%%/.23,+$4 5.3%6#22%)+/.3+()*%("#$%#+%7#11)*)+(#'28*)/3**)+/)%)93'(#.+$4 & $ !"# $ % ! % " % # & % & & Figure 9: ' $ ' % $ ' $ & $ % ! ( ' % & % % " ! ' & & & % # ' & 3 - (P3) : • 6.4 %$# ). !"# • 6 $%&'( $$ & (p.201) Eigenvalues and Eigenvectors (p.243) Systems of Differential Equations 5 (P1) (P2)

6. du(t) = Au(t), u(0) = u 0 dt u n+1 = Au n , u 0 = u 0 [ c 1 c 2 c 3 ] T , . C [ X = x 1 (λ 1 , λ 2 , λ 3 ) A X x 2 x 3 ] . u(0) = u 0 u 0 = c 1 x 1 + c 2 x 2 + c 3 x 3   c 1 c =  c 2  = X −1 u 0 c 3 : u(t) = e At u 0 = Xe Λt X −1 u 0 = Xe Λt c = c 1 e λ 1 t x 1 + c 2 e λ 2 t x 2 + c 3 e λ 3 t x 3 u n = A n u 0 = XΛ n X −1 u 0 Figure9: P3 = XΛ n c = c 1 λ n 1 x 1 + c 2 λ n 2 x 2 + c 3 λ n 3 x 3 XDc. ! !" " " !"#$%&'("')"*&+,-."/+0."%+"$")1#"+2"&$.,"3"#$%&'4-)5 $)"'.")'.617$&"8$71-9-'6-.8$71-"/-4+#:+)'%'+.; ' %! ' ! & $ & % % % ' %! ( & & % & ' ! & ! & ' % ' ' ! ' ' ! ' & % ! !"# $ % " % # % $ & & & ' Figure 10: Pattern 4 - (P4) P4 . . / , 1 . 6 . c =

7. 6 • p.vii, The Plan for the Book. A = CR, A = LU, A = QR, A = QΛQ T , A = U ΣV T . C R A = CR A A = LU A = LU ( )( ) QR A = QR - Q R S S = QΛQ T Q, A = U ΣV Λ A T Σ Table 1: 6.1 A = CR • 1.4 Matrix Multiplication and A = CR (p.29) . ). A = CR A , R ( A A . C C r A R . [ : A 1 2 A = CR ] [ ][ 3 1 2 1 0 = 5 2 3 0 1 2 3 . 1 1 ] , , . 1 . 1 2 A, C 2 2 R. " ! ! # ! " ! " " ! " " ! " " ! #$%&' !" Figure 11: CR 2, C 2 . . 7 A

8. " ! # ! ! & ' " " & & !"#$% ' ' !" Figure 12: CR , 6.2 2, 2 R , A 2 R . A = LU Ax = b . LU , (E) A U . EA = U A = E −1 U let L = E −1 , , 2 Ax = b • 2.3 : (1) A = LU Lc = b, (2) U x = c. (p.57) Matrix Computations and A = LU , A U .       0 0 0 | [ | [ ] ]  =  l 1  −u ∗ 1 − +  l 2  −u ∗ 2 − +  0 A 2 0 | | L    | [ 0 ] A =  l 1  −u ∗ 1 − +  0 | 0 # ! " " ! Figure 13: A U , L 1 A " ! 1 . A A 2 . , " Figure 14: U !"#$% " " ! 6.3  0 0  = LU A 3 . ! L 0 0 0 LU !! " A . A A = QR A = QR • 4.4 , C(A) = C(Q) A Q. Orthogonal matrices and Gram-Schmidt (p.165) - , , q 1 , a 1 a 2 q 1 q 1 = a 1 /||a 1 || q 2 = a 2 − (q 1 T a 2 )q 1 , q 3 = a 3 − (q 1 T a 3 )q 1 q 2 = q 2 /||q 2 || − (q 2 T a 3 )q 2 , 8 q 3 = q 3 /||q 3 || q 2 , .

9. r ij = q i T a j : a 1 = r 11 q 1 a 2 = r 12 q 1 + r 22 q 2 a 3 = r 13 q 1 + r 23 q 2 + r 33 q 3 A . QR:  | A =  q 1 |   | r 11 q 3   | | q 2 |  r 13 r 23  = QR r 33 r 12 r 22 QQ T = Q T Q = I ! # ! ! !"#$% " " # ! " " # ! # # !" $ # $ " $ ! " !" Figure 15: A = QR : Q A . A Q R . P1. 6.4 S = QΛQ T . S • 6.3 , Λ Q . (p.227) Symmetric Positive Definite Matrices  | S = QΛQ T =  q 1 | | q 2 |   | λ 1 q 3   |    −q 1 T −   −q 2 T −  λ 3 −q 3 T − λ 2       | [ | [ | [ ] ] ] = λ 1  q 1  −q 1 T − + λ 2  q 2  −q 2 T − + λ 3  q 3  −q 3 T − | | | = λ 1 P 1 + λ 2 P 2 + λ 3 P 3 P 1 = q 1 q 1 T , ! # " P 2 = q 2 q 2 T , # ! " $ " % # % "! ! ! # ! " ! P 3 = q 3 q 3 T # ! # $ $ % % % ! $ ! " $ & % ' % ! & " " ! $%&'( !" " Figure 16: S = QΛQ T S . , Q . 9 Λ. P = qq T

10. , P4. S = S T = λ 1 P 1 + λ 2 P 2 + λ 3 P 3 QQ T = P 1 + P 2 + P 3 = I P 1 P 2 = P 2 P 3 = P 3 P 1 = O P 1 2 = P 1 = P 1 T , 6.5 P 2 2 = P 2 = P 2 T , P 3 2 = P 3 = P 3 T A = U ΣV T • 7.1 (p.259) Singular Values and Singular Vecrtors (SVD). A = U ΣV T SVD. . Σ # % " & # ' "! $ ! " ! ! & ' ( & ! & ' , A % $ & % ' ! $ !"#$% ' !" " & U V . ' Figure 17: A = U ΣV T R n (A T A Σ. , V A  A = U ΣV T | =  u 1 | | u 2 | ) , 1   | σ 1 u 3   |  σ 2  [ −v 1 T − −v 2 T − ] R m (AA T U . ) .     | [ | [ ] ] = σ 1  u 1  −v 1 T − + σ 2  u 2  −v 2 T − | | = σ 1 u 1 v 1 T + σ 2 u 2 v 2 T : U U T = I m V V T = I n P4. / . . Ashley Fernandes , , . Linear Algebra for Everyone Gilbert Strang . . , . . 1. Gilbert Strang(2020),Linear Algebra for Everyone, Wellesley Cambridge Press., http://math.mit.edu/everyone 2. Gilbert Strang(2016), Introduction to Linear Algebra,Wellesley Cambridge Press, 5th ed., http://math.mit.edu/linearalgebra 10

11. 3. Kenji Hiranabe(2021), Map of Eigenvalues, An Agile Way(blog), https://anagileway.com/2021/10/01/map-of-eigenvalues/ Figure 18: 4. Kenji Hiranabe(2020), Matrix World, An Agile Way(blog), https://anagileway.com/2020/09/29/matrix-world-in-linear-algebra-for-everyone/ 11

12. Figure 19: 12