python 3.x - cvxpy objective function error involving MulExpression

Question

I have the code below, in it I am trying to find the maximum sharpe_ratio solution to the efficient frontier. I've adapted some code I used previously to solve the knapsack problem as an mip. I'm using the cvxpy module. When I run the code, first I get repeated warnings. then it fails with error:

"loop of ufunc does not support argument 0 of type MulExpression which has no callable sqrt method"

I've included the code below. I've also included the warning, and error. Below that I have included sample data. Can anyone see what the issue might be and suggest how to fix it?

code:

selection = cvxpy.Variable(len(weights), boolean=True)


# The sum of the weights should be less than or equal to P
weight_constraint = weights * selection <= current_account



def portfolio_performance(weights, mean_returns, cov_matrix,period):
    
    print('returns')

    returns = np.sum(weights*mean_returns.T)*period

    print('std')
    std = np.sqrt(np.dot(weights.T, np.dot(cov_matrix, weights))) * np.sqrt(period)
    return std, returns

def sharpe_ratio(weights, mean_returns, cov_matrix, risk_free_rate,period):
    p_var, p_ret = portfolio_performance(weights, mean_returns, cov_matrix,period)
    return (p_ret - risk_free_rate) / p_var




df=extended_prices

mean_returns = df.mean()
cov_matrix = df.cov()
risk_free_rate = 0.0





# Our total utility is the sum of the item utilities
# total_utility = utilities * selection

total_utility = sharpe_ratio(weights=selection, 
                             mean_returns=mean_returns, 
                             cov_matrix=cov_matrix,
                             risk_free_rate=risk_free_rate,
                            period=df.shape[0])



# We tell cvxpy that we want to maximize total utility 
# subject to weight_constraint. All constraints in 
# cvxpy must be passed as a list
knapsack_problem = cvxpy.Problem(cvxpy.Maximize(total_utility), [weight_constraint])

# Solving the problem
knapsack_problem.solve(solver=cvxpy.GLPK_MI)

warning:

UserWarning: 
This use of ``*`` has resulted in matrix multiplication.
Using ``*`` for matrix multiplication has been deprecated since CVXPY 1.1.
    Use ``*`` for matrix-scalar and vector-scalar multiplication.
    Use ``@`` for matrix-matrix and matrix-vector multiplication.
    Use ``multiply`` for elementwise multiplication.

error:

AttributeError                            Traceback (most recent call last)
AttributeError: 'MulExpression' object has no attribute 'sqrt'

The above exception was the direct cause of the following exception:

TypeError                                 Traceback (most recent call last)
<ipython-input-122-35839b05103e> in <module>
     49                              cov_matrix=cov_matrix,
     50                              risk_free_rate=risk_free_rate,
---> 51                             period=df.shape[0])
     52 
     53 

<ipython-input-122-35839b05103e> in sharpe_ratio(weights, mean_returns, cov_matrix, risk_free_rate, period)
     26 
     27 def sharpe_ratio(weights, mean_returns, cov_matrix, risk_free_rate,period):
---> 28     p_var, p_ret = portfolio_performance(weights, mean_returns, cov_matrix,period)
     29     return (p_ret - risk_free_rate) / p_var
     30 

<ipython-input-122-35839b05103e> in portfolio_performance(weights, mean_returns, cov_matrix, period)
     22 #         returns = 5
     23     print('std')
---> 24     std = np.sqrt(np.dot(weights.T, np.dot(cov_matrix, weights))) * np.sqrt(period)
     25     return std, returns
     26 

TypeError: loop of ufunc does not support argument 0 of type MulExpression which has no callable sqrt method

weights:

[171.35499572753906, 108.7699966430664, 104.56999969482422, 271.70001220703125, 253.1499938964844, 79.58999633789062, 479.6099853515625, 92.48999786376952, 152.24000549316406, 20.6299991607666, 159.17999267578125, 184.97999572753903, 41.2599983215332, 61.889997482299805, 82.5199966430664, 103.14999580383301, 123.77999496459961, 144.4099941253662, 165.0399932861328, 185.6699924468994]

sample data:

print(df[:50])

          MMM         ABT        ABBV        ABMD         ACN       ATVI  
0   175.869995  107.610001  103.809998  264.899994  248.520004  76.190002   
1   175.714996  107.355003  103.889999  265.589996  248.429993  76.690002   
2   176.135696  107.330002  104.010002  265.720001  249.085007  76.870003   
3   175.979996  107.415001  104.080002  265.910004  248.845001  77.010002   
4   176.350006  107.500000  103.849998  264.160004  249.603195  76.900002   
5   176.240005  107.245003  103.830002  262.910004  249.520004  76.629997   
6   176.700699  106.940002  103.839996  262.954987  249.100006  76.110001   
7   176.154999  106.820000  103.846497  262.595001  249.380005  76.339996   
8   176.550003  107.029999  103.955002  264.269989  249.645004  76.379997   
9   176.360001  106.809998  103.879997  263.559998  250.220001  76.430000   
10  176.804993  106.839996  103.824997  263.179993  250.139999  76.190002   
11  176.955002  106.699997  103.919998         NaN  249.520004  76.125000   
12  177.130005  106.684998  103.914597  263.209991  249.904999  76.019997   
13  176.860001  106.394997  103.769997  262.945007  249.869995  75.974998   
14  176.630005  106.500000  103.800003  262.850006  249.509995  76.070000   
15  176.580002  106.570000  103.930000  262.549988  249.770004  76.264000   
16  176.570007  106.605003  103.870003  262.399994  249.970001  76.059998   
17  176.690002  106.599998  103.860001  261.804993  249.410004  76.040001   
18  176.520004  106.790001  103.930000  261.929993  249.750000  76.074997   
19  176.845001  106.610001  103.959999  262.070007  249.279999  75.980003   
20  176.660004  106.599998  103.955002  261.575012  248.865005  76.029999   
21  176.550003  106.699997  103.870003  262.505005  248.850006  76.089996   
22  176.500000  106.660004  103.769997         NaN  248.940002  76.110001   
23  176.309998  106.669998  103.680000  262.200012  248.699997  76.250000   
24  176.315994  106.724998  103.690002  262.200012  249.020004  76.449997   
25  176.315002  106.690002  103.660004         NaN  249.065002  76.297600   
26  176.279999  106.510002  103.698997         NaN  248.809998  76.360001   
27  176.424606  106.543198  103.690002  262.595001  248.940002  76.339996   
28  176.660004  106.612900  103.739998         NaN  248.931793  76.279999   
29  176.722794  106.500000  103.660004  263.070007  249.149994  76.339996   
30  176.570007  106.419998  103.680000  263.690002  249.289993  76.535004   
31  176.686996  106.459999  103.794998  264.470001  249.330002  76.500000   
32  176.350006  106.433998  103.760002  264.260010  249.384995  76.559998   
33  176.369995  106.699997  103.629997         NaN  249.509995  76.519997   
34  176.160004  106.540001  103.709999  263.910004  249.369995  76.459999   
35  176.037598  106.332001  103.669998         NaN  249.050003  76.485001   
36  175.919998  106.379997  103.614998  264.209991  249.410004  76.480003   
37  175.899994  106.239998  103.730003  264.209991  249.429993  76.440002   
38  176.490005  106.160004  103.892700  264.200012  249.910004  76.290001   
39  176.509995  106.260002  103.949997  264.584991  249.970001  76.385002   
40  176.289993  106.459999  103.809998  264.559998  249.839996  76.440002   
41  176.619995  106.519997  103.910004         NaN  249.919998  76.320000   
42  176.350006  106.410004  103.830002  264.609985  250.009995  76.290001   
43  176.350006  106.425003  103.860001  264.540009  249.985001  76.260002   
44  176.270004  106.339996  103.870003  264.785004  249.860001  76.199898   
45  176.179993  106.360001  103.860001  264.500000  249.899994  76.106300   
46  176.205002  106.360001  103.834999  264.179993  249.759995  76.139999   
47  176.119995  106.389999  103.820000  264.519989  249.645004  76.120003   
48  176.251999  106.195000  103.800003  264.670013  249.740005  76.137802   
49  176.300003  106.239998  103.750000  264.839996  249.589996  76.114998   

          ADBE        AMD         AAP        AES      ATVI_2       AMD_2  
0   467.489990  86.769997  150.009995  21.125000  152.380005  173.539993   
1   470.070007  87.209999  149.005005  21.004999  153.380005  174.419998   
2   469.512085  87.220001  148.613098  21.000000  153.740005  174.440002   
3   468.850006  87.070000  148.529999  21.020000  154.020004  174.139999   
4   470.535004  87.550003  148.865005  21.075001  153.800003  175.100006   
5   470.410004  87.550102  148.830002  21.010000  153.259995  175.100204   
6   468.819885  87.129997  147.990005  20.995001  152.220001  174.259995   
7   469.279999  86.714897  148.070007  20.940001  152.679993  173.429794   
8   469.070007  87.078102  148.529999  20.990000  152.759995  174.156204   
9   469.459991  86.820000  148.369995  20.955000  152.860001  173.639999   
10  468.929993  86.739899  148.250000  21.010000  152.380005  173.479797   
11  466.700012  86.419998  148.190002  20.973499  152.250000  172.839996   
12  466.329987  86.339996  148.339996  21.040001  152.039993  172.679993   
13  467.734985  86.519997  148.550003  20.940001  151.949997  173.039993   
14  466.630005  86.110001  148.270004  20.940001  152.139999  172.220001   
15  467.480011  86.180000  147.669998  20.870001  152.528000  172.360001   
16  467.075012  85.889999  147.539993  20.840000  152.119995  171.779999   
17  467.070007  85.980003  147.384995  20.834999  152.080002  171.960007   
18  468.390015  86.199898  147.479996  20.875000  152.149994  172.399796   
19  467.600006  86.309998  147.570999  20.915001  151.960007  172.619995   
20  468.369995  86.520103  147.804993  20.889999  152.059998  173.040207   
21  467.644989  86.431999  147.850006  20.934999  152.179993  172.863998   
22  467.480011  86.363297  148.335007  20.934999  152.220001  172.726593   
23  467.750000  86.410004  148.735001  20.969999  152.500000  172.820007   
24  468.375000  86.529999  148.682999  21.019899  152.899994  173.059998   
25  467.540009  86.211700  148.360001  21.075001  152.595200  172.423401   
26  466.230011  86.075600  148.735001  21.080000  152.720001  172.151199   
27  466.200012  86.139397  148.820007  21.063299  152.679993  172.278793   
28  466.359985  86.121696  149.179993  21.098000  152.559998  172.243393   
29  466.440002  86.184998  149.414993  21.084999  152.679993  172.369995   
30  467.220001  86.260002  149.639999  21.090000  153.070007  172.520004   
31  466.920013  86.309998  149.770004  21.129999  153.000000  172.619995   
32  467.579987  86.364998  149.365005  21.155001  153.119995  172.729996   
33  467.890015  86.514397  149.580002  21.115000  153.039993  173.028793   
34  468.174988  86.565002  149.595001  21.075001  152.919998  173.130005   
35  467.820007  86.540001  149.330002  21.059999  152.970001  173.080002   
36  467.589996  86.660004  149.330002  21.049999  152.960007  173.320007   
37  467.530090  86.620003  149.990005  21.030001  152.880005  173.240005   
38  467.070007  86.940002  149.460007  21.030001  152.580002  173.880005   
39  466.899994  87.050003  149.740005  20.950100  152.770004  174.100006   
40  466.660004  87.124397  149.559998  20.915001  152.880005  174.248795   
41  466.899994  87.203400  149.589996  20.969999  152.639999  174.406799   
42  468.149994  87.279999  149.710007  20.969999  152.580002  174.559998   
43  469.116211  87.264999  149.839996  20.995001  152.520004  174.529999   
44  469.210785  87.235001  149.850006  20.920000  152.399796  174.470001   
45  469.052795  87.370003  149.839996  20.910000  152.212601  174.740005   
46  469.279

Answer 1

You cannot just use non-cvxpy functions on cvxpy-objects!

Some operators are overloaded (Docs) and sum / (np.sum) has it's use, but in general: don't!

Look up cvxpy's huge set of building-blocks: Atomic Functions

import cvxpy as cvx 
import numpy as np

selection = cvx.Variable(2)
np.sqrt(selection)


# AttributeError: 'Variable' object has no attribute 'sqrt'
# 
# The above exception was the direct cause of the following exception:
#
# Traceback (most recent call last):
#   File "so_cvxpy_ufunc.py", line 5, in <module>
#     np.sqrt(selection)
# TypeError: loop of ufunc does not support argument 0 of type Variable which has no 
# callable sqrt method

Compared to:

import cvxpy as cvx 

selection = cvx.Variable(2)
cvx.sqrt(selection)

# OK

Hint

You will run into other issues (fixing above won't be enough) like missing DCP-compatibility! I highly recommmend a more basic / formal / theoretical look at cvxpy first to understand what it does and how it does it to reason about it's power and limitations. I'm just saying that, as people often cycle through 5 questions here on the same problem.