Višestruka linearna regresija se primjenjuje na promjenljive koje su metričkog tipa. U praksi se često dešava da želimo da analiziramo promjenljivu koja uzima konačan broj vrijednosti i tu dolazimo do problema klasifikacije podataka. Često se javljaju promjenljive indikatorskog tipa koje uzimaju samo dvije vrijednosti, npr. muški/ženski pol, ispravan/neispravan proizvod i slično. Takvu veličinu takođe želimo da uklopimo u neki model u kojem ona zavisi od drugih promjenljivih. Ako primijenimo običnu regresiju na takav problem, dobićemo neke realne vrijednosti i teško možemo da interpretiramo takav rezultat, odnosno da kažemo kojoj klasi pripada dati podatak. Ideja je da se napravi model za vjerovatnoću \(p\) da dati indikator uzme vrijednost 1. Model (binarne) logističke regresije ima oblik:

\[ p(x)=\frac{1}{1+e^{-(\beta_0+\beta_1 x_1 +\dots +\beta_p x_P)}} \] odnosno inverz (logit funkcija):

\[ log\left(\frac{p(x)}{1-p(x)}\right)=\beta_0+\beta_1 x_1 +\dots +\beta_p x_P \] Ako se u modelu na osnovu datih prediktora dobije \(p>0.5\) onda taj podatak klasifikujemo kao “uspjeh”, u suprotnom kao “neuspjeh”.

Primjeri

baza<-read.csv("http://www.math.rs/p/files/1458635682-70-baza.txt", header=FALSE)
baza
##           V1       V2 V3
## 1   34.62366 78.02469  0
## 2   30.28671 43.89500  0
## 3   35.84741 72.90220  0
## 4   60.18260 86.30855  1
## 5   79.03274 75.34438  1
## 6   45.08328 56.31637  0
## 7   61.10666 96.51143  1
## 8   75.02475 46.55401  1
## 9   76.09879 87.42057  1
## 10  84.43282 43.53339  1
## 11  95.86156 38.22528  0
## 12  75.01366 30.60326  0
## 13  82.30705 76.48196  1
## 14  69.36459 97.71869  1
## 15  39.53834 76.03681  0
## 16  53.97105 89.20735  1
## 17  69.07014 52.74047  1
## 18  67.94686 46.67857  0
## 19  70.66151 92.92714  1
## 20  76.97878 47.57596  1
## 21  67.37203 42.83844  0
## 22  89.67678 65.79937  1
## 23  50.53479 48.85581  0
## 24  34.21206 44.20953  0
## 25  77.92409 68.97236  1
## 26  62.27101 69.95446  1
## 27  80.19018 44.82163  1
## 28  93.11439 38.80067  0
## 29  61.83021 50.25611  0
## 30  38.78580 64.99568  0
## 31  61.37929 72.80789  1
## 32  85.40452 57.05198  1
## 33  52.10798 63.12762  0
## 34  52.04540 69.43286  1
## 35  40.23689 71.16775  0
## 36  54.63511 52.21389  0
## 37  33.91550 98.86944  0
## 38  64.17699 80.90806  1
## 39  74.78925 41.57342  0
## 40  34.18364 75.23772  0
## 41  83.90239 56.30805  1
## 42  51.54772 46.85629  0
## 43  94.44337 65.56892  1
## 44  82.36875 40.61826  0
## 45  51.04775 45.82270  0
## 46  62.22268 52.06099  0
## 47  77.19303 70.45820  1
## 48  97.77160 86.72782  1
## 49  62.07306 96.76882  1
## 50  91.56497 88.69629  1
## 51  79.94482 74.16312  1
## 52  99.27253 60.99903  1
## 53  90.54671 43.39060  1
## 54  34.52451 60.39634  0
## 55  50.28650 49.80454  0
## 56  49.58668 59.80895  0
## 57  97.64563 68.86157  1
## 58  32.57720 95.59855  0
## 59  74.24869 69.82457  1
## 60  71.79646 78.45356  1
## 61  75.39561 85.75994  1
## 62  35.28611 47.02051  0
## 63  56.25382 39.26147  0
## 64  30.05882 49.59297  0
## 65  44.66826 66.45009  0
## 66  66.56089 41.09210  0
## 67  40.45755 97.53519  1
## 68  49.07256 51.88321  0
## 69  80.27957 92.11606  1
## 70  66.74672 60.99139  1
## 71  32.72283 43.30717  0
## 72  64.03932 78.03169  1
## 73  72.34649 96.22759  1
## 74  60.45789 73.09500  1
## 75  58.84096 75.85845  1
## 76  99.82786 72.36925  1
## 77  47.26427 88.47586  1
## 78  50.45816 75.80986  1
## 79  60.45556 42.50841  0
## 80  82.22666 42.71988  0
## 81  88.91390 69.80379  1
## 82  94.83451 45.69431  1
## 83  67.31926 66.58935  1
## 84  57.23871 59.51428  1
## 85  80.36676 90.96015  1
## 86  68.46852 85.59431  1
## 87  42.07545 78.84479  0
## 88  75.47770 90.42454  1
## 89  78.63542 96.64743  1
## 90  52.34800 60.76951  0
## 91  94.09433 77.15911  1
## 92  90.44855 87.50879  1
## 93  55.48216 35.57070  0
## 94  74.49269 84.84514  1
## 95  89.84581 45.35828  1
## 96  83.48916 48.38029  1
## 97  42.26170 87.10385  1
## 98  99.31501 68.77541  1
## 99  55.34002 64.93194  1
## 100 74.77589 89.52981  1
# Baza sadrzi rezultate studenata na prijemnim ispitima na nekom fakultetu za upis na master studije (prve dvije kolone)
# dok treca je indikator da li je dati student upisao ili nije.

Primjećujemo da promjenljiva \(V3\) uzima vrijednosti 0 i 1.

attach(baza)
plot(V3~V1+V2)

model.log <- glm(V3 ~ V1 + V2, family = binomial)
summary(model.log)
## 
## Call:
## glm(formula = V3 ~ V1 + V2, family = binomial)
## 
## Deviance Residuals: 
##      Min        1Q    Median        3Q       Max  
## -2.19287  -0.18009   0.01577   0.19578   1.78527  
## 
## Coefficients:
##              Estimate Std. Error z value Pr(>|z|)    
## (Intercept) -25.16133    5.79836  -4.339 1.43e-05 ***
## V1            0.20623    0.04800   4.297 1.73e-05 ***
## V2            0.20147    0.04862   4.144 3.42e-05 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 134.6  on 99  degrees of freedom
## Residual deviance:  40.7  on 97  degrees of freedom
## AIC: 46.7
## 
## Number of Fisher Scoring iterations: 7
# Vrijednosti za p dobijene modelom

V3.p <- fitted(model.log)

# Klasifikujemo na osnovu p

V3.klas <- ifelse(V3.p >= 0.5, 1, 0)

# Nekada je pozeljno reci samo da sa nekom vjerovatnocom podatak pripada odredjenoj klasi.
# Na primjer, ako je p=0.5 nemamo preciznu odluku.

# Brojimo koliko je puta model "pogodio"

mean(V3 == V3.klas) # u 89% slucajeva je pogodio
## [1] 0.89
# Sada hocemo dobijeni model da iskoristimo za predvidjanje. Neka je dato da je student osvojio 
# na prijemnom V1=20 i V2=100 poena. Hocemo da procjenimo da li ce upisati master.

predict(model.log, newdata =  data.frame(V1 = 20, V2 = 100), type = "response")
##         1 
## 0.2912049

Primjer 2

library(ISLR)

baza <- Smarket

names(baza)
## [1] "Year"      "Lag1"      "Lag2"      "Lag3"      "Lag4"      "Lag5"     
## [7] "Volume"    "Today"     "Direction"
# 
# Year
# The year that the observation was recorded
# 
# Lag1
# Percentage return for previous day
# 
# Lag2
# Percentage return for 2 days previous
# 
# Lag3
# Percentage return for 3 days previous
# 
# Lag4
# Percentage return for 4 days previous
# 
# Lag5
# Percentage return for 5 days previous
# 
# Volume
# Volume of shares traded (number of daily shares traded in billions)
# 
# Today
# Percentage return for today
# 
# Direction
# A factor with levels Down and Up indicating whether the market had a positive or negative return on a given day


attach(baza)

Direction[1:10]
##  [1] Up   Up   Down Up   Up   Up   Down Up   Up   Up  
## Levels: Down Up
mdl <- glm(Direction ~ Lag1 + Lag2 + Lag3 + Lag4 + Lag5 + Volume, family =
             binomial)
             
summary(mdl)
## 
## Call:
## glm(formula = Direction ~ Lag1 + Lag2 + Lag3 + Lag4 + Lag5 + 
##     Volume, family = binomial)
## 
## Deviance Residuals: 
##    Min      1Q  Median      3Q     Max  
## -1.446  -1.203   1.065   1.145   1.326  
## 
## Coefficients:
##              Estimate Std. Error z value Pr(>|z|)
## (Intercept) -0.126000   0.240736  -0.523    0.601
## Lag1        -0.073074   0.050167  -1.457    0.145
## Lag2        -0.042301   0.050086  -0.845    0.398
## Lag3         0.011085   0.049939   0.222    0.824
## Lag4         0.009359   0.049974   0.187    0.851
## Lag5         0.010313   0.049511   0.208    0.835
## Volume       0.135441   0.158360   0.855    0.392
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 1731.2  on 1249  degrees of freedom
## Residual deviance: 1727.6  on 1243  degrees of freedom
## AIC: 1741.6
## 
## Number of Fisher Scoring iterations: 3
mdl.p <- fitted(mdl)

# Ako je p>=0.5, dodjeljujemo "Up", inace "Down"

mdl.fit <- ifelse(mdl.p > 0.5, "Up", "Down")

mean(mdl.fit == Direction)
## [1] 0.5216
# Sada hocemo da pravimo model na osnovu poduzorka iz date baze, odnosno
# izdvojicemo one podatke do 2005 godine i na osnovu njih napraviti model, a
# potom predvidjati vrijednosti na osnovu preostalog uzorka

uzorak1 <- Year<2005
mdl1 <- glm(
  Direction ~ Lag1 + Lag2 + Lag3 + Lag4 + Lag5 + Volume,
  data = Smarket,
  family = binomial,
  subset = uzorak1
  )
  summary(mdl1)
## 
## Call:
## glm(formula = Direction ~ Lag1 + Lag2 + Lag3 + Lag4 + Lag5 + 
##     Volume, family = binomial, data = Smarket, subset = uzorak1)
## 
## Deviance Residuals: 
##    Min      1Q  Median      3Q     Max  
## -1.302  -1.190   1.079   1.160   1.350  
## 
## Coefficients:
##              Estimate Std. Error z value Pr(>|z|)
## (Intercept)  0.191213   0.333690   0.573    0.567
## Lag1        -0.054178   0.051785  -1.046    0.295
## Lag2        -0.045805   0.051797  -0.884    0.377
## Lag3         0.007200   0.051644   0.139    0.889
## Lag4         0.006441   0.051706   0.125    0.901
## Lag5        -0.004223   0.051138  -0.083    0.934
## Volume      -0.116257   0.239618  -0.485    0.628
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 1383.3  on 997  degrees of freedom
## Residual deviance: 1381.1  on 991  degrees of freedom
## AIC: 1395.1
## 
## Number of Fisher Scoring iterations: 3
  mdl1.p <- predict(mdl1, newdata = Smarket[!uzorak1, ], type = "response")
  
  mdl1.fit <- ifelse(mdl1.p > 0.5, "Up", "Down")
  Direction.2005 <- Direction[!uzorak1]
  
  # Provjeravamo koliko je model pogodio
  
  table(mdl1.fit, Direction.2005)
##         Direction.2005
## mdl1.fit Down Up
##     Down   77 97
##     Up     34 44
  mean(mdl1.fit == Direction.2005)
## [1] 0.4801587
# Probamo model sa manje prediktora
mdl2 <- glm(Direction~Lag1+Lag2,
            data=Smarket,family=binomial, subset=uzorak1)
mdl2.p <- predict(mdl2 ,newdata=Smarket[!uzorak1,],type="response") 
mdl2.fit <- ifelse(mdl2.p > 0.5, "Up", "Down")


table(mdl2.fit, Direction.2005)
##         Direction.2005
## mdl2.fit Down  Up
##     Down   35  35
##     Up     76 106
mean(mdl2.fit == Direction.2005)
## [1] 0.5595238