Loans Receivable - Recorded Past Due (Details) (USD $)
In Thousands, unless otherwise specified
Dec. 31, 2013
Dec. 31, 2012
PR
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current $ 2,788,327 $ 3,079,717
30 to 89 Days Past Due 254,324 202,687
Loans 90 days or more past due 583,646 666,911
Loans over 90 days past due and still accruing 4,193 7,660
US
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 2,712,611 2,220,003
30 to 89 Days Past Due 87 263
Loans 90 days or more past due 3,034 5,229
Loans over 90 days past due and still accruing 0 0
Residential Mortgage and Other Consumer Loan [Member] | PR
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 2,417,726 2,661,977
30 to 89 Days Past Due 172,211 150,185
Loans 90 days or more past due 368,151 380,036
Loans over 90 days past due and still accruing 3,407 7,247
Residential Mortgage and Other Consumer Loan [Member] | US
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 9,520 11,716
30 to 89 Days Past Due 87 263
Loans 90 days or more past due 778 201
Loans over 90 days past due and still accruing 0 0
Residential Mortgage [Member] | PR
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 2,389,570 2,627,687
30 to 89 Days Past Due 166,561 145,589
Loans 90 days or more past due 336,407 334,549
Loans over 90 days past due and still accruing 0 0
Residential Mortgage [Member] | US
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 9,344 11,677
30 to 89 Days Past Due 87 263
Loans 90 days or more past due 778 201
Loans over 90 days past due and still accruing 0 0
Fha Va Guaranteed Residential Mortgage [Member] | PR
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 10,673 11,668
30 to 89 Days Past Due 4,646 4,090
Loans 90 days or more past due 30,967 43,941
Loans over 90 days past due and still accruing 2,692 6,129
Fha Va Guaranteed Residential Mortgage [Member] | US
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 0 0
30 to 89 Days Past Due 0 0
Loans 90 days or more past due 0 0
Loans over 90 days past due and still accruing 0 0
Consumer Loan [Member] | PR
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 17,483 22,622
30 to 89 Days Past Due 1,004 506
Loans 90 days or more past due 777 1,546
Loans over 90 days past due and still accruing 715 1,118
Consumer Loan [Member] | US
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 176 39
30 to 89 Days Past Due 0 0
Loans 90 days or more past due 0 0
Loans over 90 days past due and still accruing 0 0
Commercial Loan [Member] | PR
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 370,601 417,740
30 to 89 Days Past Due 82,113 52,502
Loans 90 days or more past due 215,495 286,875
Loans over 90 days past due and still accruing 786 413
Commercial Loan [Member] | US
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 2,703,091 2,208,287
30 to 89 Days Past Due 0 0
Loans 90 days or more past due 2,256 5,028
Loans over 90 days past due and still accruing 0 0
Commercial Real Estate [Member] | PR
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 213,447 259,661
30 to 89 Days Past Due 77,397 47,004
Loans 90 days or more past due 129,522 172,831
Loans over 90 days past due and still accruing 0 0
Commercial Real Estate [Member] | US
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 697,916 630,923
30 to 89 Days Past Due 0 0
Loans 90 days or more past due 646 646
Loans over 90 days past due and still accruing 0 0
Commercial and Industrial Loan [Member] | PR
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 118,915 124,176
30 to 89 Days Past Due 375 867
Loans 90 days or more past due 1,343 5,761
Loans over 90 days past due and still accruing 786 413
Commercial and Industrial Loan [Member] | US
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 1,482,146 1,420,918
30 to 89 Days Past Due 0 0
Loans 90 days or more past due 0 0
Loans over 90 days past due and still accruing 0 0
Borrowings on Land and Construction [Member] | PR
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 38,239 33,903
30 to 89 Days Past Due 4,341 4,631
Loans 90 days or more past due 84,630 108,283
Loans over 90 days past due and still accruing 0 0
Borrowings on Land and Construction [Member] | US
    
Financing Receivable, Recorded Investment, Past Due [Line Items]    
Current 523,029 156,446
30 to 89 Days Past Due 0 0
Loans 90 days or more past due 1,610 4,382
Loans over 90 days past due and still accruing $ 0 $ 0