All Questions
Tagged with probability numpy
209
questions
72
votes
10
answers
117k
views
Multivariate normal density in Python?
Is there any python package that allows the efficient computation of the PDF (probability density function) of a multivariate normal distribution?
It doesn't seem to be included in Numpy/Scipy, and ...
- 5,376
30
votes
9
answers
13k
views
How to do weighted random sample of categories in python
Given a list of tuples where each tuple consists of a probability and an item I'd like to sample an item according to its probability. For example, give the list [ (.3, 'a'), (.4, 'b'), (.3, 'c')] I'd ...
- 303
28
votes
3
answers
76k
views
Defining a white noise process in Python
I need to draw samples from a white noise process in order to implement a particular integral numerically.
How do I generate this with Python (i.e., numpy, scipy, etc.)?
- 10.5k
23
votes
2
answers
21k
views
Fitting distributions, goodness of fit, p-value. Is it possible to do this with Scipy (Python)?
INTRODUCTION: I'm a bioinformatician. In my analysis which I perform on all human genes (about 20 000) I search for a particular short sequence motif to check how many times this motif occurs in each ...
- 2,347
23
votes
1
answer
3k
views
sampling multinomial from small log probability vectors in numpy/scipy
Is there a function in numpy/scipy that lets you sample multinomial from a vector of small log probabilities, without losing precision? example:
# sample element randomly from these log probabilities
...
- 1,472
14
votes
3
answers
15k
views
How to generate random numbers with predefined probability distribution?
I would like to implement a function in python (using numpy) that takes a mathematical function (for ex. p(x) = e^(-x) like below) as input and generates random numbers, that are distributed according ...
- 1,884
13
votes
2
answers
4k
views
multinomial pmf in python scipy/numpy
Is there a built-in function in scipy/numpy for getting the PMF of a Multinomial? I'm not sure if binom generalizes in the correct way, e.g.
# Attempt to define multinomial with n = 10, p = [0.1, 0.1,...
user248237
12
votes
10
answers
19k
views
How do I simulate biased die in python?
I want to simulate N-sided biased die?
def roll(N,bias):
'''this function rolls N dimensional die with biasing provided'''
# do something
return result
>> N=6
>> bias=( 0....
- 36.3k
11
votes
1
answer
52k
views
Distribution plot of an array
I have a numpy array containing float values in [-10..10]. I would like to plot a distribution-graph of the values, like this (here it is done for a binomial random variable) :
For example I would ...
- 43.8k
8
votes
4
answers
5k
views
Computing a binomial probability for huge numbers
I want to compute binomial probabilities on python. I tried to apply the formula:
probability = scipy.misc.comb(n,k)*(p**k)*((1-p)**(n-k))
Some of the probabilities I get are infinite. I checked ...
- 2,143
8
votes
2
answers
11k
views
Generating random numbers with a given probability density function
I want to specify the probability density function of a distribution and then pick up N random numbers from that distribution in Python. How do I go about doing that?
- 107
8
votes
4
answers
6k
views
Calculating pdf of Dirichlet distribution in python
I'd like to calculate the pdf for the Dirichlet distribution in python, but haven't been able to find code to do so in any kind of standard library. scipy.stats includes a long list of distributions ...
- 115
8
votes
1
answer
215
views
Probability that a formula fails in IEEE 754
On my computer, I can check that
(0.1 + 0.2) + 0.3 == 0.1 + (0.2 + 0.3)
evaluates to False.
More generally, I can estimate that the formula (a + b) + c == a + (b + c) fails roughly 17% of the time ...
7
votes
3
answers
24k
views
Calculate moments (mean, variance) of distribution in python
I have two arrays. x is the independent variable, and counts is the number of counts of x occurring, like a histogram. I know I can calculate the mean by defining a function:
def mean(x,counts):
...
- 415
7
votes
3
answers
8k
views
How to properly sample truncated distributions?
I am trying to learn how to sample truncated distributions. To begin with I decided to try a simple example I found here example
I didn't really understand the division by the CDF, therefore I ...
- 798
7
votes
4
answers
4k
views
Python equivalent for MATLAB's normplot?
Is there a python equivalent function similar to normplot from MATLAB?
Perhaps in matplotlib?
MATLAB syntax:
x = normrnd(10,1,25,1);
normplot(x)
Gives:
I have tried using matplotlib & numpy ...
- 2,135
7
votes
1
answer
7k
views
How can I sample a multivariate log-normal distribution in Python?
Using Python, how can I sample data from a multivariate log-normal distribution? For instance, for a multivariate normal, there are two options. Let's assume we have a 3 x 3 covariance matrix and a 3-...
- 2,654
6
votes
2
answers
316
views
How can I generate random numbers given I have probability of varous ranges specified in python
I want to fill in the dummy data of attendance. I want that,for example, 60% students have attendance in the range of 70-100,25% in the range of 40-60 and 15% in the range of 0-40. How can I generate ...
5
votes
3
answers
699
views
Speeding up normal distribution probability mass allocation
We have N users with P avg. points per user, where each point is a single value between 0 and 1. We need to distribute the mass of each point using a normal distribution with a known density of 0.05 ...
- 5,685
5
votes
2
answers
8k
views
How to draw a sample from a categorical distribution
I have a 3D numpy array with the probabilities of each category in the last dimension. Something like:
import numpy as np
from scipy.special import softmax
array = np.random.normal(size=(10, 100, 5))
...
- 394
5
votes
3
answers
4k
views
Generating random numbers with weighted probabilities in python
Given a positive integer array a, the goal is to generate 5 random numbers based on the weight they have in the array.
For example:
a = [2,3,4,4,4,4,4,6,7,8,9]
In this case the number 4 has ...
- 729
5
votes
1
answer
6k
views
Best way to get joint probability from 2D numpy
Was wondering if there is a better way to get the probability of a 2D numpy array. Maybe using some of numpy's built in functions.
For simplicity, say we have an example array:
[['apple','pie'],
['...
- 2,803
4
votes
1
answer
12k
views
Python: Matplotlib - probability plot for several data set
I have several data sets (distribution) as follows:
set1 = [1,2,3,4,5]
set2 = [3,4,5,6,7]
set3 = [1,3,4,5,8]
How do I plot a scatter plot with the data sets above with the y-axis being the ...
- 2,135
4
votes
1
answer
2k
views
About how numpy.random.choice works
I'm trying to writing a program that simulate the Coupon collector's problem. Here is quick reference for the problem:
Given n coupons, how many coupons do you expect you need to draw with
...
- 193
4
votes
1
answer
1k
views
How is scipy.stats.multivariate_normal.pdf different from the same function written using numpy?
I need to use the multivariate normal distribution in a script. I have noticed that my version of it gives a different answer from scipy's method. I can't really figure out why...
Here is my function:...
- 43
4
votes
4
answers
252
views
How to probabilistically populate a list in python?
I want to use a basic for loop to populate a list of values in Python but I would like the values to be calculate probabilistically such that p% of the time the values are calculated in (toy) equation ...
- 463
4
votes
2
answers
2k
views
Calculate probability 2 random people are in the same group?
In my dataset, there are N people who are each split into one 3 groups (groups = {A, B, C}). I want to find the probability that two random people, n_1 and n_2, belong to the same group.
I have data ...
- 313
4
votes
2
answers
5k
views
How to draw a random sample from a Poisson distribution?
I have a number X of integers (very large) and a probability p with which I want to draw a sample s (a number) from X following a Poisson distribution. For example, if X = 10^8 and p=0.05, I expect s ...
- 49
3
votes
3
answers
2k
views
Uniformly generating random pairs of different integers
Task:
Generate a pair of random numbers (i,j) (order doesn't matter: (i,j) is equivalent to (j,i)).
The pair must consist of two distinct values: i != j
Pairs must be uniformly distributed. In other ...
- 133
3
votes
2
answers
14k
views
Numpy - correlation coefficient and related statistical functions don't give same results
For data X = [0,0,1,1,0]and Y = [1,1,0,1,1]
>> np.corrcoef(X,Y)
returns
array([[ 1. , -0.61237244],
[-0.61237244, 1. ]])
However, I cannot reproduce this result using ...
- 1,243
3
votes
3
answers
897
views
Binomial distributions (Bernoulli trials) with different probabilities
I want to speed up the code below - namely the for loop. Is there a way to do it in numpy?
import numpy as np
# define seend and random state
rng = np.random.default_rng(0)
# num of throws
N = 10**1
# ...
- 2,323
3
votes
1
answer
3k
views
Building N-th order Markovian transition matrix from a given sequence
I am trying to create a function which can transform a given input sequence to a transition matrix of the requested order. I found an implementation for the first-order Markovian transition matrix.
...
- 252
3
votes
2
answers
4k
views
python sampling from different distributions with different probability
I am trying to implement a fucntion which returns 100 samples from three different multivariate gaussian distributions.
numpy provides a way to sample from a sinle multivariate gaussian. But I could ...
- 1,596
3
votes
2
answers
4k
views
Numpy Histogram Representing Floats with Approximate Values as The Same
I have code that generates a certain value from -10 to 10 given a range from [0,1)
The code takes the value from -10 to 10 and it will append it to a list, according to its probability. For example, -...
- 579
3
votes
2
answers
6k
views
how to calculate the estimated joint probability distribution for the red and green pixels only in a png image?
I have a png image that contains the red and green channels only. I removed the blue channel from the image for calculation purposes. I need to calculate the estimated joint probability distribution ...
- 131
3
votes
1
answer
2k
views
calculate binomial distribution probability matrix with python
Give the N and P, I want to get a 2D binomial distribution probability matrix M,
for i in range(1, N+1):
for j in range(i+1):
M[i,j] = choose(i, j) * p**j * (1-p)**(i-j)
other value = 0
I ...
- 43
3
votes
3
answers
2k
views
probability of T total eyes when throwing N dice with S sides
I want to calculate the probability of the event that the sum of all eyes of n dice with s sides (numbered from 1 to s) is equal to t. My language is Python 3.
My current approach is pretty much a ...
- 6,610
3
votes
1
answer
2k
views
Python Custom Zipf Number Generator Performing Poorly
I needed a custom Zipf-like number generator because numpy.random.zipf function doesn't achieve what I need. Firstly, its alpha must be greater than 1.0 and I need an alpha of 0.5. Secondly, its ...
- 3,023
3
votes
2
answers
2k
views
Generating a binomial distribution around zero
I'm looking to generate a binomial-esque distribution. I want a binomial distribution but I want it centred around zero (I know this doesn't make much sense with respect to the definition of binomial ...
- 1,231
3
votes
2
answers
2k
views
Compute probability over a multivariate normal
My question addresses both mathematical and CS issues, but since I need a performant implementation I am posting it here.
Problem:
I have an estimated normal bivariate distribution, defined as a ...
- 555
3
votes
2
answers
476
views
Choosing random number where probability is random in Python
While I can find decent information on how to generate numbers based on probabilities for picking each number with numpy.random.choice e.g.:
np.random.choice(5, 3, p=[0.1, 0, 0.3, 0.6, 0])
which ...
- 1,657
3
votes
2
answers
740
views
Implementing the generalized birthday paradox in Python
My question is about numerical problems I am running into when implementing a probability function, and not about the probability/mathematics behind it. I'm also aware that my code below is probably ...
- 175
3
votes
1
answer
9k
views
Probability distribution function in Python
I know how to create an histogram in Python, but I would like that it is the probability density distribution.
Let's start with my example. I have an array d, with a size of 500000 elements. With the ...
- 14.5k
3
votes
1
answer
2k
views
How to sample from a log-probability distribution?
I have some code that uses log-probability. When I want to draw a sample from the probability distribution, I use
import numpy as np
probs = np.exp(logprobs)
probs /= probs.sum()
sample = np.random....
- 71
3
votes
2
answers
563
views
Multiple ngrams in transition matrix, probability not adding to 1
I'm trying to find a way to make a transition matrix using unigrams, bigrams, and trigrams for a given text using python and numpy. Each row's probabilities should equal to one. I did this first with ...
- 162
3
votes
0
answers
428
views
Calculate the likelihood of a function given a Gaussian Process model
I am fitting a Gaussian process regression using scikit-learn. (mine is actually a simple 1 dimensional case)
from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn....
- 2,705
2
votes
2
answers
1k
views
Understanding Markov Chains in terms of Matrix Multiplication
In a lecture on YouTube, a professor said Markov Chains could be simplified to Start(S) * Transition Matrix(Q)^State#
I'm trying to replicate this using numpy.
import numpy as np
S = np.zeros(shape=(...
- 17.3k
2
votes
2
answers
6k
views
Efficient random 0,1 generation of independant probabilities in Python
I have a large array of probabilities that are independent of each other. Is there an efficient way to generate a 0 or 1 based on each of these probabilities other than calling numpy random each time ...
- 1,355
2
votes
2
answers
3k
views
How to compute and plot the pdf from the empirical cdf?
I have two numpy arrays, one is an array of x values and the other an array of y values and together they give me the empirical cdf. E.g.:
plt.plot(xvalues, yvalues)
plt.show()
I assume the data ...
- 20.6k
2
votes
3
answers
3k
views
Discretization of probability array in Python
I have a numpy array (actually imported from a GIS raster map) which contains
probability values of occurrence of a species like following example:
a = random.randint(1.0,20.0,1200).reshape(40,30)
b =...
- 1,034