CN111225050B - Cloud computing resource allocation method and device - Google Patents
Cloud computing resource allocation method and device Download PDFInfo
- Publication number
- CN111225050B CN111225050B CN202010002687.8A CN202010002687A CN111225050B CN 111225050 B CN111225050 B CN 111225050B CN 202010002687 A CN202010002687 A CN 202010002687A CN 111225050 B CN111225050 B CN 111225050B
- Authority
- CN
- China
- Prior art keywords
- resource
- target task
- node
- matching degree
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1012—Server selection for load balancing based on compliance of requirements or conditions with available server resources
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computer Hardware Design (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application relates to a cloud computing resource allocation method, a cloud computing resource allocation device, cloud computing resource allocation equipment and a storage medium. A cloud computing resource allocation method is based on a target task, the matching degree of each node in a cloud system is obtained, and the target task is allocated to the node with the highest matching degree. The matching degree is obtained according to the sub-matching degree corresponding to each resource, and the sub-matching degree is obtained according to the demand of the target task on the resource, the weight of the resource and the surplus of the resource in the node. When the node with the highest matching degree corresponding to the target task is determined, the target task is allocated to the node with the highest matching degree, each target task is sequentially allocated to the corresponding node with the highest matching degree, and dynamic resource allocation of each target task is achieved, so that resources in the cloud system can be fully utilized under the condition that the demand of each target task on each resource can be met, meanwhile, the number of target tasks which can be executed by the system in unit time is increased, and the resource use efficiency of the system is optimized.
Description
Technical Field
The present application relates to the field of cloud computing technologies, and in particular, to a method and an apparatus for allocating cloud computing resources.
Background
With the rapid development of cloud computing, a cloud computing system includes resources of multiple dimensions, including a Central Processing Unit (CPU), a memory, a network bandwidth, a local disk bandwidth, and the like, which are in continuous dynamic changes, and the performance of the cloud computing system is affected by the allocation and management of each resource.
In the implementation process, the inventor finds that at least the following problems exist in the conventional technology: in a traditional resource allocation scheme of the cloud computing system, only the completion time required by the cloud task is considered, and the resource allocation factor is single, so that the resource utilization efficiency of the cloud computing system is limited.
Disclosure of Invention
Therefore, it is necessary to provide a cloud computing resource allocation method and apparatus for solving the problem of low resource utilization rate of the conventional cloud computing system.
In order to achieve the above object, in one aspect, an embodiment of the present application provides a cloud computing resource allocation method, including:
based on the target task, obtaining the matching degree of each node in the cloud system; the matching degree is obtained according to the sub-matching degree corresponding to each resource; the sub-matching degree is obtained according to the demand of the target task on the resources, the weight of the resources and the residual quantity of the resources in the nodes;
and allocating the target task to the node with the highest matching degree.
In one embodiment, before the step of obtaining the matching degree of each node in the cloud system based on the target task, the method further includes the steps of:
determining the weight of each resource according to the average demand of each resource; the average demand of the resources is obtained according to the demand of each task in the cloud system on the resources.
In one embodiment, before the step of determining the weight of each resource according to the average demand of each resource, the method further includes the steps of:
the average demand of the resource is obtained based on the following formula:
wherein, avdem t Represents the average demand, of the t-th resource i，t The method comprises the steps of representing the demand of a target task i to a t-th resource, wherein N is the target task number needing the t-th resource, i =1,2, \8230, m, t =1,2, \8230, and N is the resource type number.
In one embodiment, the step of determining the weight of each resource according to the average demand of each resource further includes the steps of:
the weight of the resource is obtained based on the following formula:
w t ＝exp(∈*avdem t )
wherein the content of the first and second substances,w t as a weight of the resource, avdem t And e is the average demand quantity of the resource, and is the experience weight factor corresponding to the resource.
In one embodiment, before the step of obtaining the matching degree of each point in the cloud system, the method further includes the steps of:
acquiring the sub-matching degree of the resource based on the following formula:
F t ＝R[i，t]*C[j，t]*w t
wherein, F t Is the sub-match degree of the t-th resource, R [ i, t ]]The demand for the t-th resource for the target task i, cj, t]Is the residual amount of the t-th resource in the node j, w t I =1,2, \ 8230;, m, t =1,2, \ 8230;, n, j =1,2, \ 8230;, p, m is the target task number, n is the resource type number, and p is the node number.
In one embodiment, the step of obtaining the matching degree according to the sub-matching degree corresponding to each resource further includes the steps of:
the matching degree is obtained based on the following formula:
F i，j ＝∑ t F t
wherein, F i，j Is the matching degree of the target task i corresponding to the node j, F t For the sub-matching degree of the t-th resource, i =1,2, \8230;, m, t =1,2, \8230;, n, j =1,2, \8230;, p, m is the target task number, n is the resource kind number, and p is the node number.
In one embodiment, the method further comprises the following steps:
when a new node is added to the cloud system, the dispersion of each target task in the cloud system is obtained based on the new node; the dispersion is obtained according to the sub-dispersion corresponding to each resource; the sub-dispersion is obtained according to the demand of the target task on the resource, the weight of the resource and the surplus of the resource in the new node;
and allocating the target task with the minimum divergence to the new node.
On the other hand, an embodiment of the present application further provides a cloud computing resource allocation apparatus, including:
the matching degree obtaining module is used for obtaining the matching degree of each node in the cloud system based on the target task; the matching degree is obtained according to the sub-matching degree corresponding to each resource; the sub-matching degree is obtained according to the demand of the target task on the resources, the weight of the resources and the residual quantity of the resources in the nodes;
and the target task allocation module is used for allocating the target task to the node with the highest matching degree.
One of the above technical solutions has the following advantages and beneficial effects:
based on the target task, obtaining the matching degree of each corresponding node in the cloud system, wherein the matching degree of the node is obtained according to the sub-matching degree corresponding to each resource; and the sub-matching degree is obtained by calculation according to the resource demand of the target task, the weight of the resource and the surplus of the resource in the node. When the node with the highest matching degree corresponding to the target task is obtained, the node is regarded as the node matched with the required quantity of the target task to each resource, the target task is distributed to the node with the highest matching degree, each target task is sequentially distributed to the corresponding node with the highest matching degree, and dynamic resource distribution of each target task is achieved, so that the resources in the system can be fully utilized under the condition that the required quantity of the target task to each resource can be met, meanwhile, the number of the target tasks which can be executed by the system through limited resources in unit time is increased, the use efficiency of the system resources is optimized, and the processing performance of the cloud system is improved.
Drawings
The foregoing and other objects, features and advantages of the application will be apparent from the following more particular description of preferred embodiments of the application, as illustrated in the accompanying drawings. Like reference numerals refer to like parts throughout the drawings, and the drawings are not intended to be drawn to scale in actual size, with emphasis instead being placed upon illustrating the subject matter of the present application.
FIG. 1 is a first schematic flow chart diagram of a method for cloud computing resource allocation in one embodiment;
FIG. 2 is a second schematic flow chart diagram of a method for cloud computing resource allocation in one embodiment;
FIG. 3 is a comparison of the case where the method of the present application is not used and the case where the method of the present application is used to allocate resources to 5 target tasks in one embodiment;
fig. 4 is a schematic structural diagram of a cloud computing resource allocation apparatus in one embodiment.
Detailed Description
To facilitate an understanding of the present application, the present application will now be described more fully with reference to the accompanying drawings. Preferred embodiments of the present application are shown in the drawings. This application may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
It will be understood that when an element is referred to as being "connected" to another element, it can be directly connected to the other element and be integral therewith, or intervening elements may also be present. The terms "including," "having," and the like, as used herein, are used for descriptive purposes only.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
The embodiment of the application can be applied to the field of cloud computing; the core idea of cloud computing is to uniformly manage and allocate a large number of resources connected by a network to form a computing resource pool to serve users as required. At present, a cloud computing system comprises a plurality of dynamic resources, including but not limited to a CPU, a memory, a network bandwidth, a local disk bandwidth, and the like, which are in continuous dynamic change, a resource allocation scheme of a conventional cloud computing system usually only considers completion time required by a cloud task, and a resource allocation factor is single, so that cloud computing resource utilization efficiency is low, and tasks that can be executed by a system with low resource utilization rate through limited resources in unit time are very limited, thereby limiting performance of the cloud computing system.
In one embodiment, as shown in fig. 1, a cloud computing resource allocation method is provided, which may be applied in a cloud computing system and a data warehouse, and may be implemented by a device or a server for resource allocation.
A cloud computing resource allocation method, comprising:
step S210, acquiring the matching degree of each node in the cloud system based on the target task; the matching degree is obtained according to the sub-matching degree corresponding to each resource; the sub-matching degree is obtained according to the resource demand of the target task, the weight of the resource and the residual quantity of the resource in the node.
And step S220, distributing the target task to the node with the highest matching degree.
Specifically, the matching degree of each node in the cloud system and the target task is obtained. And distributing the target task to the node with the highest matching degree. The matching degree of each node corresponding to the target task in the cloud system is obtained, the node with the highest matching degree is regarded as the node which is most matched with the target task, the target task is distributed to the node with the highest matching degree, then the surplus of each resource in the node with the highest matching degree is distributed to the target task according to the demand of the target task on the corresponding resource, and then each target task is sequentially distributed to the corresponding node with the highest matching degree, so that the dynamic distribution of each resource is realized, under the condition that the demand of the target task on each resource can be met, the number of target tasks which can be executed in unit time by the cloud system is increased, and the resources of the cloud system are utilized to the maximum extent. Optionally, the resource includes at least one of a CPU core count, a CPU peak, a memory, a cache capacity, a network bandwidth, a local disk bandwidth, and a process count.
Specifically, the matching degree is obtained according to the sub-matching degree corresponding to each resource. And the sub-matching degree is obtained according to the demand of the target task on the resources, the weight of the resources and the residual quantity of the resources in the nodes, and reflects the matching degree of the demand of the target task on the resources and the residual quantity of the resources in the nodes. For example, the matching degree of the node may be the sum of the sub-matching degrees of the node corresponding to the resources and/or the average of the sub-matching degrees of the node corresponding to the resources. The weight of each resource reflects the real-time resource demand condition, because tasks at different moments in the cloud computing system have different task attributes, sometimes a CPU sensitive task is taken as a main task, sometimes a memory sensitive task is taken as a main task, in order to better reflect the real-time resource demand condition, the weight of each resource in the cloud computing system needs to be obtained, the matching degree of each node is further obtained according to the weight of each resource, the target task is distributed to the node with the highest matching degree according to the matching degree of the target task and each node, efficient dynamic resource distribution is achieved, resource shortage or surplus is avoided, and further maximization of resource utilization rate is achieved.
Illustratively, the target tasks are marked as target task 1, target task 2 \8230andtarget task n. The resources required by the target task 1 comprise memory resources, network bandwidth resources and CPU resources, the matching degree of each node in the cloud system is obtained, the target task 1 is distributed to the node with the highest matching degree, and each resource required correspondingly is distributed to the target task 1. And respectively obtaining the matching degree corresponding to the node through the sub-matching degree corresponding to the memory resource in the node, the sub-matching degree corresponding to the network broadband resource and the sub-matching degree corresponding to the CPU resource, and further obtaining the matching degree of each node. The sub-matching degree corresponding to the memory resource in the node is obtained according to the demand of the target task 1 on the corresponding memory resource, the weight of the memory resource and the residual amount of the memory resource in the corresponding node; the sub-matching degree corresponding to the network bandwidth resources in the nodes is obtained according to the demand of the target task 1 on the network bandwidth resources, the weight of the network bandwidth resources and the residual amount of the network bandwidth resources in the corresponding nodes; the sub-matching degree corresponding to the CPU resource in the node is obtained according to the demand of the target task 1 on the CPU resource, the weight of the CPU resource and the residual amount of the CPU resource in the corresponding node. And distributing each target task to each corresponding node with the highest matching degree, and realizing dynamic distribution of each resource through resource distribution of each resource pair corresponding to each task in the node with the highest matching degree, so that the utilization rate of the cloud system resource is improved.
In an embodiment, as shown in fig. 2, before the step of obtaining the matching degree of each node in the cloud system based on the target task, the method further includes the steps of:
step S200, determining the weight of each resource according to the average demand of each resource; the average demand of the resources is obtained according to the demand of each task in the cloud system on the resources.
Specifically, since target tasks at different times in the cloud computing system have different task attributes, sometimes CPU-sensitive tasks are mainly used, and sometimes memory-sensitive tasks are mainly used, in the resource allocation process, to better reflect the real-time resource demand condition, weights of various resources in the computing system are required. And obtaining the average demand of the resources according to the demand of each target task on the resources in the cloud system, and determining the weight corresponding to each resource according to the average demand corresponding to each resource in the cloud system, thereby obtaining the corresponding sub-matching degree according to the weight of each resource. Illustratively, the total demand of the resource is obtained by calculating and accumulating the demands of each target task on the resource, the ratio of the total demand of the resource to the number of the target tasks is determined as the average demand of the resource, the average demand corresponding to each resource is further calculated, and the weight corresponding to each resource is determined according to the average demand corresponding to each resource in the cloud system.
In one embodiment, before the step of determining the weight of each resource according to the average demand of each resource, the method further includes the steps of:
the average demand of the resource is obtained based on the following formula:
wherein, avdem t Indicating the average demand, of the t-th resource i，t The method comprises the steps of representing the demand of a target task i to a t-th resource, wherein N is the target task number needing the t-th resource, i =1,2, \8230, m, t =1,2, \8230, and N is the resource type number.
Specifically, i =1,2, \ 8230;, m, t =1,2, \ 8230;,n，demand i，t demand for t-th resource of target task i, avdem t Representing the average demand of the t-th resource, and controlling the demand of the task i on the t-th resource i，t The product value of the sum and the reciprocal of the resource variety N is determined as the average demand avdem of the t resource t And N is the target task number to be considered, so the average demand of the resources can represent the average demand of each target task on the t-th resources.
In one embodiment, the step of determining the weight of each resource according to the average demand of each resource further includes the steps of:
the weight of the resource is obtained based on the following formula:
wt＝exp(∈*avdem t )
wherein, w t As a weight of the resource, avdem t And e is the average demand quantity of the resources, and the epsilon is an experience weight factor corresponding to the resources.
In particular, the weight w of the resource t Average demand avdem corresponding to resource t The empirical weight factor belonging to the resource is obtained, the weight corresponding to a certain resource can be determined according to the formula, and the obtained weight can more accurately represent the proportion of the resource in all resources, so that the requirement condition of implementing the resource can be better reflected, and the efficient dynamic allocation is realized. Illustratively, the memory resource corresponds to a resource weight of 0.55.
In one embodiment, before the step of obtaining the matching degree of each point in the cloud system, the method further includes the steps of:
acquiring the sub-matching degree of the resource based on the following formula:
F t ＝R[i，t]*C[j，t]*w t
wherein, F t Is the sub-match degree of the t-th resource, R [ i, t ]]The demand for the t-th resource for the target task i, cj, t]Is the residual amount of the t-th resource in the node j, w t I =1,2, \ 8230;, m, t =1,2, \ 8230;, n, j =1,2, \ 8230;, p, m is the target task number, n is the resource type number, and p is the node number.
Specifically, i =1,2, \8230;, m, t =1,2, \8230;, n, j =1,2, \8230;, p, acquires the demand R [ i, t ] for the t-th resource by the target task i]Weight w of the t-th resource t And the residual quantity C [ j, t of the t-th resource in the corresponding node j]Is determined as the sub-matching degree F of the t-th resource in the corresponding node t And further, obtaining the matching degree of the target task and the corresponding node j according to the sub-matching degree corresponding to each resource in the corresponding node j.
In one embodiment, the step of obtaining the matching degree according to the sub-matching degree corresponding to each resource further includes the steps of:
obtaining the matching degree based on the following formula:
F i，j ＝∑ t F t
wherein, F i，j Is the matching degree of the target task i corresponding to the node j, F t For the sub-matching degree of the t-th resource, i =1,2, \8230;, m, t =1,2, \8230;, n, j =1,2, \8230;, p, m is the target task number, n is the resource kind number, and p is the node number.
Specifically, F t Denotes the sub-match degree of the t-th resource, where t =1,2, \8230;, n, and F i，j For the degree of matching corresponding to node j for target task i, i =1,2, \8230;, m, j =1,2, \8230;, p. Matching degree F corresponding to each resource t The sum is determined as the matching degree F corresponding to the target task i and the node j i，j . According to the matching degree of each node and the target task, the target task is allocated to the node with the highest matching degree, and each target task is sequentially allocated to the corresponding node with the highest matching degree, so that dynamic resource allocation of each target task is realized, the resource requirement of each target task is met, the resources in the system can be fully utilized, and the resource utilization rate of the cloud system is improved.
In one embodiment, the step of assigning the target task to the node with the highest matching degree further includes the steps of:
and (4) sequencing the matching degree of each node from high to low, and distributing the target task to the node with the highest matching degree.
Specifically, the matching degrees of the nodes corresponding to the target tasks are ranked from high to low, the target tasks are allocated to the nodes with the highest matching degree, the target tasks are continuously allocated to the nodes with the highest matching degree, the process is repeated until dynamic resources are dynamically allocated to the target tasks, the resources in the system can be fully utilized under the condition that the demand of the target tasks for the resources can be met, the number of tasks which can be executed by the cloud system through the limited resources in unit time is increased, and the use efficiency of the system resources is optimized. Illustratively, the size of the memory provided by a certain node is 4G, as shown in fig. 3, the execution conditions of 5 target tasks in the cloud computing resource allocation method are compared respectively by using a FIFO (First Input First Output) algorithm, the abscissa represents time, and the ordinate represents resource usage.
According to the FIFO algorithm, 1 target task with 3G memory requirement of the first arriving node is executed in the first time slice, 1 target task with 2G memory requirement of the second arriving node is executed in the second time slice, and 3 target tasks with 1G memory requirement of the last arriving node are executed in the third time slice, 3 time slices are spent totally, and the node has unequal memory resource waste in the 3 time slices. According to the cloud computing resource allocation method, in a first time slice, the initial residual amount of the memory resources in the node is 4G, and the average demand amount of the memory resources is computed firstly
Weight w of memory resource Memory 1 ＝exp(ε*avdem Memory 1 ) And (∈ 1.6), and obtaining the matching degree F of the node corresponding to the target task 1 1，j ＝∑ Memory device R1, internal memory]* Cj memory]*w Memory 1 ＝3*4*w Memory 1 And sequentially calculating the matching degrees corresponding to the target tasks 2, 3, 4 and 5, and distributing the target task 1 in the first time slice because the matching degree of the target task 1 is the highest at the moment. The residual amount of the memory resource of the first time slice becomes 1G, and the second timeThe residual amount of the memory resources of the slice is still 4G, the sub-matching degree corresponding to the memory resources is obtained in real time according to the weight of the memory resources, the demand of the unallocated target tasks on the memory resources and the residual amount of the memory resources in each time slice, and further the matching degree of the corresponding nodes and each unallocated target task is obtained. Calculating the matching degree of the nodes and the unallocated target tasks in real time and the average demand of memory resources
w Memory 2 ＝exp(ε*avdem Memory 2 ) = exp (∈ 1), and the matching degree F corresponding to each time slice and the target task 2 is obtained 2，j ＝∑ Memory device R2, internal memory]* Cj memory]*w Memory 2 The matching degree of the target task 2 and the second time slice is the highest, and the target task 2 is distributed in the second time slice; and calculating according to the same steps, wherein the matching degree of the target task 3 and the first time slice is the highest, distributing the target task 3 into the first time slice, and repeating the operation until 5 target tasks are distributed completely, so that 2 time slices are spent totally. It can be seen that, when the cloud computing resource allocation method is adopted to execute 5 task allocation, only 2 time slices are needed, the resource allocation time is shortened, on the premise that each target task obtains the required resource, the resource waste of the node in the time slices is avoided, and meanwhile, the number of tasks running in unit time of the cloud system is increased.
In one embodiment, further comprising:
when a new node is added to the cloud system, the dispersion of each target task in the cloud system is obtained based on the new node; the dispersion is obtained according to the sub-dispersion corresponding to each resource; the sub-dispersion is obtained according to the demand of the target task on the resource, the weight of the resource and the surplus of the resource in the new node;
and allocating the target task with the minimum divergence to the new node.
Specifically, when a new node is added to the cloud system, the dispersion between each target task and the new node in the cloud system is obtained based on the new node, the target task corresponding to the minimum dispersion is selected to be allocated to the new node, repeated operation is continuously performed to allocate as many target tasks as possible to the new node, and therefore the execution speed of the target tasks is accelerated, the resource utilization rate of the new node is maximized, and dynamic resource allocation based on the dispersion of each node is achieved. The dispersion is obtained according to the sub-dispersion corresponding to each resource; the sub-dispersion is obtained according to the demand of the target task on the resource, the weight of the resource and the residual quantity of the resource in the new node. Illustratively, the weight of each resource is derived based on the following formula:
w t ＝exp(∈*avdem t )
wherein, w t Is the weight of the t-th resource, avdem t And e is the average demand of the t-th resource, and t =1,2, \ 8230;, n is the empirical weighting factor corresponding to the t-th resource.
In one embodiment, the step of obtaining the dispersion according to the sub-dispersion corresponding to each resource further includes the steps of:
obtaining a sub-dispersion of the resource based on the following formula:
varience t ＝w t *(R[i，t]-C[j，t])
wherein, the development t Is the sub-dispersion of the t-th resource, R [ i, t ]]The demand for the t-th resource for the target task i, cj, t]Is the residual amount of the t-th resource in the new node j, w t I =1,2, \ 8230;, m, t =1,2, \ 8230;, n, j =1,2, \ 8230;, p, m are the number of target tasks, n is the number of resource types, and p is the number of nodes.
Specifically, i =1,2, \ 8230;, m, t =1,2, \ 8230;, n, j =1,2, \ 8230;, p, varience t And obtaining the sub-dispersion corresponding to the target task i and the t-th resource, wherein the sub-dispersion is obtained by the demand of the target task i on the t-th resource, the residual quantity of the t-th resource in the new node j and the weight of the t-th resource.
In one embodiment, the step of obtaining the dispersion according to the sub-dispersion corresponding to each resource further includes the steps of:
obtaining the dispersion based on the following formula:
variance i ＝∑ t variance t
wherein, the variance i Variance for target task i t For the sub-dispersion of the t-th resource, i =1,2, \ 8230;, m, t =1,2, \ 8230;, n, m is the number of target tasks, and n is the number of resource types.
Specifically, i =1,2, \8230;, m, t =1,2, \8230;, n, variance i Is the dispersion of the target task i, i.e. the dispersion of the target task i and the new node j, variance i And accumulating the sub-dispersion of each resource corresponding to the target task i. And then, the dispersion degree of each target task and a newly-added node j in the cluster is sorted from large to small, and the target task with the minimum dispersion degree is distributed to the new node. And then after the task with the minimum dispersion is distributed, calculating the dispersion of the remaining target tasks in the waiting queue and the new node again, continuously distributing the target task with the minimum dispersion to the new node, and repeating the process until the remaining resources in the new node cannot run more tasks. According to the embodiment of the application, when new nodes are added to the cloud system, the new nodes can be distributed with as many target tasks as possible, so that the execution speed of the target tasks in the waiting queue is accelerated, the resource utilization rate of the new nodes is maximized, the resource distribution efficiency is improved, and multi-dimensional resource distribution in the cloud system is realized. In one embodiment, k =1, 2.. Multidot.m, r =1, 2.. Multidot.k, when there are k target tasks whose dispersion is the minimum, one task is selected from the k target tasks to be allocated, wherein the probability that the r-th task is selected is (1-p) k And the value of p is determined according to the number of the target tasks.
In one example, if a new node j joins an original cluster, the new node j does not run any target task temporarily, the size of the memory resource provided by the new node j is 8G, and the size of the cache resource is 6G. Meanwhile, three target tasks waiting for distribution are respectively a target task a, a target task b and a target task c in the waiting queue, wherein the target task a needs a memory resource 1G and a cache resource 2G; the target task b needs a memory resource 2G and a cache resource 2G; and the target task c needs a memory resource 2G and a cache resource 3G.
First, the average demand of memory resources at the first time is calculated
Average demand of cache resources
Further calculating to obtain the weight of the memory resource
Weight of cache resources
Respectively calculating the dispersion of the target task a, the target task b, the target task c and the newly added node j by using the resource weight: variance a，1 ＝w Memory 1 *(8-1) 2 +w Cache 1 *(6-2) 2 ，variance b，1 ＝w Memory 1 *(8-2) 2 +w Cache 1 *(6-2) 2 ，variance c，1 ＝w Memory 1 *(8-2) 2 +w Cache 1 *(6-3) 2 . It can be known that the dispersion of the task c and the available resources in the newly added node j is the minimum at this time, so the task c is allocated to the newly added node j in the first step.
After the target task c is allocated, the remaining situation of the newly added node j resource is as follows: if the memory resources are remained for 6G and the cache resources are remained for 3G, recalculating the average demand of each remaining resource in the newly added node j, wherein the average demand of the memory resources is
Average demand of cache resources
Corresponding memory resource weight
Cache resource weight w Cache 2 And = exp (∈ 2), the dispersion of the target task a, the target task b, and the newly added node j at this time can be obtained: variance is provided a，2 ＝w Memory 2 *(6-1) 2 +w Cache 2 *(3-2) 2 ，variance b，2 ＝w Memory 2 *(6-2) 2 +w Cache 2 *(3-2) 2 。
And calculating to obtain the minimum dispersion of the target task b and the available resources in the newly added node at the moment, so that the target task b is distributed to the newly added node j at the step. And repeating the processes until the newly added node has no residual resources to run more tasks. Based on this, according to the embodiment of the application, the target task with the minimum dispersion with the newly added node j at the same time can be allocated to the newly added node through the dispersion of each target task and the newly added node j, and the target tasks are sequentially allocated to the newly added node until the residual resources in the newly added node are not enough to run more target tasks, so that as many tasks as possible are allocated to the newly added node, the resource allocation efficiency in a cloud system is improved, various requirements of different resource dimensions on a resource allocation mode in actual application are met, and multi-dimensional elastic resource allocation is realized.
In one embodiment, the resources include CPU, memory, cache capacity, network bandwidth, local disk bandwidth, and process count.
It should be understood that although the various steps in the flow charts of fig. 1-2 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 1-2 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 4, there is provided a cloud computing resource allocation apparatus, including:
the matching degree obtaining module is used for obtaining the matching degree of each node in the cloud system based on the target task; the matching degree is obtained according to the sub-matching degree corresponding to each resource; the sub-matching degree is obtained according to the demand of the target task on the resources, the weight of the resources and the residual quantity of the resources in the nodes;
and the target task allocation module is used for allocating the target tasks to the nodes with the highest matching degree.
Specifically, the cloud computing resource allocation device acquires the matching degree of each node in the cloud system through the matching acquisition module, the matching degree is acquired according to the sub-matching degree corresponding to each resource, the sub-matching degree is acquired according to the quantity of the target task required for the resource, the weight of the resource and the residual quantity of the resource in the node, and then allocates each target task to the corresponding node with the highest matching degree through the target task allocation module.
In one embodiment, the degree of match module includes:
the resource weight determining unit is used for determining the weight of each resource according to the average demand of each resource; the average demand of the resources is obtained according to the demand of each task in the cloud system on the resources.
For specific limitations of the cloud computing resource allocation apparatus, reference may be made to the above limitations of the cloud computing resource allocation method, which is not described herein again. It should be noted that, in the embodiment of the present application, the division of the module is schematic, and is only one logic function division, and there may be another division manner in actual implementation. The modules in the cloud computing resource allocation apparatus may be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, there is provided a computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
based on the target task, obtaining the matching degree of each node in the cloud system; the matching degree is obtained according to the sub-matching degree corresponding to each resource; the sub-matching degree is obtained according to the resource demand of the target task, the weight of the resource and the residual quantity of the resource in the node.
And allocating the target task to the node with the highest matching degree.
In one embodiment, a computer storage medium is provided, having a computer program stored thereon, the computer program, when executed by a processor, implementing the steps of:
based on the target task, obtaining the matching degree of each node in the cloud system; the matching degree is obtained according to the sub-matching degree corresponding to each resource; the sub-matching degree is obtained according to the resource demand of the target task, the weight of the resource and the residual quantity of the resource in the node.
And allocating the target task to the node with the highest matching degree.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.
Claims (8)
1. A cloud computing resource allocation method, comprising:
obtaining the average demand of the resources according to the demand of each task in the cloud system on the resources; the average demand of the resource is obtained based on the following formula:
wherein, avdem t Represents the average demand, of the t-th resource i，t The method comprises the steps of representing the demand of a target task i on a t-th resource, wherein N is the target task number needing the t-th resource, i =1,2, \8230, m, t =1,2, \8230, N, m is the target task number, and N is the resource type number;
determining the weight of each resource according to the average demand of the resource and the experience weight factor corresponding to each resource;
based on the target task, obtaining the matching degree of each node in the cloud system; the matching degree is obtained by summing the sub-matching degrees corresponding to the resources; the sub-matching degree is obtained by multiplying the required quantity of the target task to the resource, the weight of the resource and the residual quantity of the resource in the node;
and distributing the target task to the node with the highest matching degree.
2. The cloud computing resource allocation method according to claim 1, wherein the step of determining the weight of each resource according to the average demand amount of the resource and an empirical weight factor corresponding to each resource further comprises the steps of:
obtaining the weight of the resource based on the following formula:
w t ＝exp(∈*avdem t )
wherein, w t For said weight of said resource, avdem t And e is the average demand of the resource, and is an empirical weighting factor corresponding to the resource.
3. The method for allocating cloud computing resources of claim 1, wherein before the step of obtaining the matching degree of each node in the cloud system, the method further comprises the steps of:
obtaining the sub-matching degree of the resource based on the following formula:
F t ＝R[i，t]*C[j，t]*w t
wherein, F t For the sub-match degree of the t-th resource, R [ i, t ]]The demand of said t-th resource for said target task i, cj, t]Is the residual amount of said t-th resource in node j, w t I =1,2, \ 8230;, m, t =1,2, \ 8230;, n, j =1,2, \ 8230;, p, m is the target task number, n is the resource kind number, and p is the node number, for the weight of the t-th resource.
4. The cloud computing resource allocation method according to claim 3, wherein the matching degree is obtained by summing up the sub-matching degrees corresponding to the respective resources, and the method further comprises:
obtaining the matching degree based on the following formula:
F i，j ＝∑ t F t
wherein, F i，j The matching degree, F, corresponding to the target task i and the node j t I =1,2, \ 8230for the sub-matching degree of the t-th resource, m, t =1,2, \ 8230, n, j =1,2, \ 8230, p, m is the target task number, n is the resource kind number, and p is the node number.
5. The cloud computing resource allocation method according to claim 1, further comprising:
when a new node is added to the cloud system, obtaining the dispersion of each target task in the cloud system based on the new node; the dispersion is obtained according to the sub-dispersion corresponding to each resource; the sub-dispersion is obtained according to the demand of the target task on the resource, the weight of the resource and the residual quantity of the resource in the new node;
and allocating the target task with the minimum divergence to the new node.
6. The cloud computing resource allocation method according to claim 5, wherein the step of obtaining the dispersion according to the sub-dispersion corresponding to each resource further includes the steps of:
acquiring the sub-dispersion of the resource based on the following formula:
varience t ＝w t *(R[i，t]-C[j，t])
wherein, the variance t Is the sub-dispersion of the t-th resource, R [ i, t ]]The demand for the t-th resource for the target task i, cj, t]Is the residual amount of the t-th resource in the new node j, w t I =1,2, \ 8230;, m, t =1,2, \ 8230;, n, j, the weight of the t-th resource=1,2, \8230, p, m is the target task number, n is the resource kind number, and p is the node number.
7. The cloud computing resource allocation method according to claim 6, wherein the step of obtaining the dispersion according to the sub-dispersion corresponding to each of the resources further includes the steps of:
obtaining the dispersion based on the following formula:
variance i ＝∑ t variance t
wherein, the variance i Variance for target task i t The sub-dispersion of the t resource, i =1,2, \8230;, m, t =1,2, \8230;, n, m is the target task number, and n is the resource type number.
8. A cloud computing resource allocation apparatus, comprising:
the resource weight determining unit is used for obtaining the average demand of the resources according to the demand of each task in the cloud system on the resources; the average demand of the resource is obtained based on the following formula:
wherein, avdem t Represents the average demand, of the t-th resource i，t The method comprises the steps of representing the demand of a target task i on a t-th resource, wherein N is the target task number needing the t-th resource, i =1,2, \8230, m, t =1,2, \8230, N, m is the target task number, and N is the resource type number;
the matching degree acquisition module is used for determining the weight of each resource according to the average demand of the resource and the experience weight factor corresponding to each resource; based on the target task, obtaining the matching degree of each node in the cloud system; the matching degree is obtained by summing the sub-matching degrees corresponding to the resources; the sub-matching degree is obtained by multiplying the required quantity of the target task to the resource, the weight of the resource and the residual quantity of the resource in the node;
and the target task allocation module is used for allocating the target task to the node with the highest matching degree.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010002687.8A CN111225050B (en) | 2020-01-02 | 2020-01-02 | Cloud computing resource allocation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010002687.8A CN111225050B (en) | 2020-01-02 | 2020-01-02 | Cloud computing resource allocation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111225050A CN111225050A (en) | 2020-06-02 |
| CN111225050B true CN111225050B (en) | 2022-10-18 |
Family
ID=70832235
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010002687.8A Active CN111225050B (en) | 2020-01-02 | 2020-01-02 | Cloud computing resource allocation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111225050B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111831454B (en) * | 2020-09-17 | 2021-02-19 | 北京汇智恒安科技有限责任公司 | Cloud computing server cluster resource allocation control system and method |
| CN112288227A (en) * | 2020-09-25 | 2021-01-29 | 长沙市到家悠享网络科技有限公司 | Task management method, device and equipment |
| CN116361018A (en) * | 2021-12-27 | 2023-06-30 | 格步计程车控股私人有限公司 | Method and equipment for distributing parallel flow tasks based on dynamic resource state |
| CN114860672B (en) * | 2022-07-01 | 2023-03-24 | 苏州优鲜信网络生活服务科技有限公司 | Node management method and system for batch processing data task |
| CN115242662B (en) * | 2022-09-22 | 2023-02-17 | 音信云(武汉)信息技术有限公司 | Data resource allocation method and device based on cloud computing |
| CN117240773B (en) * | 2023-11-15 | 2024-02-02 | 华北电力大学 | Method, device, equipment and medium for arranging nodes of power communication network |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104301944A (en) * | 2014-09-17 | 2015-01-21 | 华为技术有限公司 | Resource capacity allocation method and equipment |
| CN105141517A (en) * | 2015-10-13 | 2015-12-09 | 重庆邮电大学 | Flexible optical network anycast business energy-saving routing method based on resource perception |
| CN105847453A (en) * | 2016-06-22 | 2016-08-10 | 中国联合网络通信集团有限公司 | Service level real-time adjustment method and device |
| CN106874116A (en) * | 2017-02-14 | 2017-06-20 | 郑州云海信息技术有限公司 | A kind of server selection method and device for placing newly-built virtual machine |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI456502B (en) * | 2011-12-01 | 2014-10-11 | Univ Tunghai | Dynamic resource allocation method for virtual machine cluster |
| TW201324365A (en) * | 2011-12-07 | 2013-06-16 | Ind Tech Res Inst | Method of resource allocation and resource arbitrator |
| US8468251B1 (en) * | 2011-12-29 | 2013-06-18 | Joyent, Inc. | Dynamic throttling of access to computing resources in multi-tenant systems |
| CN103369041B (en) * | 2013-07-09 | 2017-10-03 | 北京奇虎科技有限公司 | Resource allocation methods and device based on cloud computing |
| CN109254842B (en) * | 2017-07-12 | 2023-06-16 | 腾讯科技（深圳）有限公司 | Resource management method and device for distributed stream system and readable storage medium |
| CN107770096B (en) * | 2017-12-11 | 2021-07-30 | 国网河南省电力公司信息通信公司 | SDN/NFV network dynamic resource allocation method based on load balancing |
| CN108469988B (en) * | 2018-02-28 | 2021-12-17 | 西北大学 | Task scheduling method based on heterogeneous Hadoop cluster |
| CN110187960A (en) * | 2019-04-23 | 2019-08-30 | 广东省智能制造研究所 | A kind of distributed resource scheduling method and device |
-
2020
- 2020-01-02 CN CN202010002687.8A patent/CN111225050B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104301944A (en) * | 2014-09-17 | 2015-01-21 | 华为技术有限公司 | Resource capacity allocation method and equipment |
| CN105141517A (en) * | 2015-10-13 | 2015-12-09 | 重庆邮电大学 | Flexible optical network anycast business energy-saving routing method based on resource perception |
| CN105847453A (en) * | 2016-06-22 | 2016-08-10 | 中国联合网络通信集团有限公司 | Service level real-time adjustment method and device |
| CN106874116A (en) * | 2017-02-14 | 2017-06-20 | 郑州云海信息技术有限公司 | A kind of server selection method and device for placing newly-built virtual machine |
Non-Patent Citations (1)
| Title |
|---|
| More than bin packing:Dynamic resource allocation strategies in cloud data centers;Andreas Wolke;《Information Systems》;20151231;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111225050A (en) | 2020-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111225050B (en) | Cloud computing resource allocation method and device | |
| US11496413B2 (en) | Allocating cloud computing resources in a cloud computing environment based on user predictability | |
| CN103701886A (en) | Hierarchic scheduling method for service and resources in cloud computation environment | |
| CN109710406B (en) | Data distribution and model training method and device thereof, and computing cluster | |
| CN111861412B (en) | Completion time optimization-oriented scientific workflow scheduling method and system | |
| CN109799956B (en) | Memory controller and IO request processing method | |
| CN110633135A (en) | Asynchronous task allocation method and device, computer equipment and storage medium | |
| CN104881322A (en) | Method and device for dispatching cluster resource based on packing model | |
| CN111176840A (en) | Distributed task allocation optimization method and device, storage medium and electronic device | |
| CN111190696A (en) | Docker container deployment method, system, device and storage medium | |
| Moulik et al. | COST: A cluster-oriented scheduling technique for heterogeneous multi-cores | |
| CN111796933A (en) | Resource scheduling method, device, storage medium and electronic equipment | |
| CN107370783B (en) | Scheduling method and device for cloud computing cluster resources | |
| CN108388409B (en) | Print request processing method, apparatus, computer device and storage medium | |
| CN116302404B (en) | Resource decoupling data center-oriented server non-perception calculation scheduling method | |
| Guo | Ant colony optimization computing resource allocation algorithm based on cloud computing environment | |
| CN112181498A (en) | Concurrency control method, device and equipment | |
| CN110178119B (en) | Method, device and storage system for processing service request | |
| CN115562841B (en) | Cloud video service self-adaptive resource scheduling system and method | |
| WO2020088078A1 (en) | Fpga-based data processing method, apparatus, device and medium | |
| Ray et al. | Is high performance computing (HPC) ready to handle big data? | |
| Behrouzi-Far et al. | Data replication for reducing computing time in distributed systems with stragglers | |
| CN115827178A (en) | Edge calculation task allocation method and device, computer equipment and related medium | |
| CN112306642B (en) | Workflow scheduling method based on stable matching game theory | |
| Al-Yatama et al. | Memory allocation algorithm for cloud services |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |