SEGMENT INFORMATION (Details) - USD ($)
$ in Thousands
 3 Months Ended 12 Months Ended
Dec. 31, 2016
Sep. 30, 2016
Jun. 30, 2016
Mar. 31, 2016
Dec. 31, 2015
Sep. 30, 2015
Jun. 30, 2015
Mar. 31, 2015
Dec. 31, 2016
Dec. 31, 2015
Dec. 31, 2014
Segment Reporting Information [Line Items]                      
Net interest income (expense) $ 25,959 $ 27,265 $ 27,989 $ 24,041 $ 26,452 $ 25,437 $ 28,317 $ 24,701 $ 105,254 $ 104,907 $ 64,598
Provision for loan losses 2,445 1,392 629 1,125 1,052 6,699 2,379 997 5,591 11,127 92
Noninterest income 30,486 14,937 14,016 12,618 12,799 14,464 14,197 18,022 72,057 59,482 20,441
Noninterest expense 34,093 28,663 30,903 27,639 27,692 27,823 30,703 31,546 121,298 117,764 69,480
Income (loss) before income taxes (benefit) 19,907 12,147 10,473 7,895 10,507 5,379 9,432 10,180 50,422 35,498 15,467
Income taxes (benefit) 8,327 4,102 3,683 2,777 2,811 1,928 2,762 3,590 18,889 11,091 4,651
Net income (loss) 11,580 $ 8,045 $ 6,790 $ 5,118 7,696 $ 3,451 $ 6,670 $ 6,590 31,533 24,407 10,816
Total assets 3,233,723       2,884,824       3,233,723 2,884,824 2,676,614
Banking                      
Segment Reporting Information [Line Items]                      
Net interest income (expense)                 109,950 107,825 66,449
Provision for loan losses                 5,591 11,127 92
Noninterest income                 47,646 34,751 13,343
Noninterest expense                 100,062 95,712 63,351
Income (loss) before income taxes (benefit)                 51,943 35,737 16,349
Income taxes (benefit)                 17,129 10,626 5,181
Net income (loss)                 34,814 25,111 11,168
Total assets 3,225,765       2,886,539       3,225,765 2,886,539 2,710,476
Commercial FHA Origination and Servicing                      
Segment Reporting Information [Line Items]                      
Net interest income (expense)                 1,003 1,762  
Noninterest income                 22,979 20,148  
Noninterest expense                 15,104 16,232  
Income (loss) before income taxes (benefit)                 8,878 5,678  
Income taxes (benefit)                 3,332 2,271  
Net income (loss)                 5,546 3,407  
Total assets 129,943       89,242       129,943 89,242 119,649
Other                      
Segment Reporting Information [Line Items]                      
Net interest income (expense)                 (5,699) (4,680) (1,851)
Noninterest income                 1,432 4,583 7,098
Noninterest expense                 6,132 5,820 6,129
Income (loss) before income taxes (benefit)                 (10,399) (5,917) (882)
Income taxes (benefit)                 (1,572) (1,806) (530)
Net income (loss)                 (8,827) (4,111) (352)
Total assets $ (121,985)       $ (90,957)       $ (121,985) $ (90,957) $ (153,511)