The�Motion Magnification�algorithm can magnify small motions in videos in specific bands of frequencies; for structures if you choose the frequencies to coincide with resonant frequencies you can create videos of the approximate or operational mode shapes. Expanding on this, using phase based optical flow, quantitative displacements can be measured from videos and mode shapes can be identified.
Currently, in pseudo real-time we can take a video of a small structure, identify and measure its operational mode shapes, and generate motion magnified video at the identified resonant frequencies. Vibrational inspection of small structures and machinery is currently being developed. For measurements of distant civil infrastructure, issues such as atmospheric turbulence and camera platform motion will be investigated.
J. G. Chen, N. Wadhwa, Y.-J. Cha, F. Durand, W. T. Freeman and O. Buyukozturk. Modal Identification of Simple Structures with High-Speed Video using Motion Magnification. Journal of Sound and Vibration 2015; 345: 58-71. [highlighted by MIT News]
J. G. Chen, T. M. Adams, H. Sun, E. S. Bell and O. Buyukozturk. Camera-based vibration measurement of the Portsmouth, NH WWI Memorial Bridge. Journal of Structural Engineering, ASCE 2018; 144(11); 04018207.
The structural health monitoring research community has recently started the application of image processing techniques for vision-based monitoring of infrastructure systems. Such techniques typically involve extensive image manipulation and feature extraction for reliable damage detection. However, variablity of real-world scenarios, like lighting and shadow changes, hinders the applicability of such algorithms for field monitoring. To overcome these challenges, a deep convolutional neural network (CNN)-based framework employed for damage detection instead, which has demonstrated its efficacy in dealing with real-world scenarios.
Y-J. Cha, W. Choi and O. Buyukozturk. Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering 2017; 32(5): 361-378.
Y‐J. Cha, W. Choi, G. Suh, S. Mahmoudkhani and O. Büyüköztürk. Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types. Computer‐Aided Civil and Infrastructure Engineering 2017; DOI:10.1111/mice.12334.
Modelling complex spatiotemporal dynamical systems, such as reaction-diffusion processes, which can be found in many fundamental dynamical effects in various disciplines, has largely relied on finding the underlying partial differential equations (PDEs). However, predicting the evolution of these systems remains a challenging task for many cases owing to insufficient prior knowledge and a lack of explicit PDE formulation for describing the nonlinear process of the system variables. With recent data-driven approaches, it is possible to learn from measurement data while adding prior physics knowledge. However, existing physics-informed machine learning paradigms impose physics laws through soft penalty constraints, and the solution quality largely depends on a trial-and-error proper setting of hyperparameters. In this work, a deep learning framework that forcibly encodes a given physics structure in a recurrent convolutional neural network is developed to facilitate learning of the spatiotemporal dynamics in sparse data regimes. The proposed physics-encoding machine learning approach shows high accuracy, robustness, interpretability and generalizability (see Figure 3).
C. Rao, P. Ren, O. Buyukozturk, H. Sun, and Y. Liu. Encoding physics to learn reaction-diffusion processes. Nature Machine intelligence 2023, 5, 765-779.
Vibration-based structural health monitoring (SHM) provides a primary tool for evaluating structural condition, integrity and reliability as well as for assessing potential risks throughout the lifecycle of structures. The objective of this research is to propose an integrated sensing and decision-making system for SHM of tall buildings (see Figure 4). This research addresses challenges related to computational modeling, vibration sensor optimal deployment, model updating and data analytics including mining, identification, uncertainty quantification, etc. A continuous monitoring system can build a temporal database for the building and detect possible damage due to structural deterioration or extreme events at an early stage.
Computational modeling, in addition to data analytics, plays an important role in structural health monitoring (SHM). A high fidelity computational model based on teh design and construction information provides important dynamic characteristics of a structure. Additionally, it may be used for devising sensor placement strategies, and based on acquired field measurements it can be updated for SHM puposes such as damage detection, response prediction and reliability assessment. The following figures show the finite element model developed for the world's tallest sculptured RC tower, Al Hamra building in Kuwait City, Kuwait.
H. Sun, J. Al-Qazweeni, J. Parol, H. Kamal, Z. Chen and O. Buyukozturk. Computational modeling of a unique tower in Kuwait for structural health monitoring: numerical investigations. Structural Control & Health Monitoring 2019; 26(3): e2317
The objective of optimal sensor placement (OSP) is to obtain a sensor layout that gives as much information of the dynamic system as possible in SHM. The process of OSP can be formulated as a discrete minimization (or maximization) problem with the sensor locations as the design variables, conditional on the constraint of a given sensor number. A finite element model-based optimal sensor placement algorithm is investigated for building instrumentation and has been successfully validated on the Green Building (83.47 m) at MIT shown in Figure 5.
H. Sun and O. Buyukozturk. Optimal sensor placement in structural health monitoring using discrete optimization. Smart Materials and Structures 2015; 24(12): 125034.
Continuous monitoring of buildings provides a crucial alternative to assess its health condition as well as evaluate its safety throughout the whole service life. The vibration of a building is related to the excitation, the soil-structure interaction, and the building mechanical properties. Separating the building response from the excitation and the soil-structure interaction using vibrational data yields information of the intrinsic characteristics of the building. A deconvolution-based a seismic interferometric method is proposed to extract the shear waves propagating in the building (see Figure 6). Intrinsic physical properties such as wave velocity, mode shapes and damping ratios can be continuously extracted if the building is monitored temporally (see Figure 7). The identified building properties can then be used for structural condition evaluation, model updating, damage detection, etc.
H. Sun, A. Mordret, G. A. Prieto, N. Toksoz and O. Buyukozturk. Bayesian characterization of buildings using seismic interferometry on ambient vibrations. Mechanical Systems and Signal Processing 2017; 85: 468-486. [Highlighted on MIT News]
Quantification of the uncertain parameters of a building model is important in SHM since it is related to assessing the health condition, evaluating the integrity, and estimating the capacity to carry loads and risk. The objective of this study is to establish a numerical model, calibrated against field records, for building response prediction subjected to potential extreme events and for damage detection in future operations of the building. A hierarchical Bayesian inference approach is investigated to quantify the uncertainties of the model parameters. For example, the distributions of storey stiffnesses can be continuously updated over time and used for possible building damage detection after an extreme event such as earthquake (see Figure 8).
Prediction of structural response is necessary for evaluating condition and quantifying vulnerability of structural systems exposed to seismic loads. Traditional modeling techniques for infrastructure systems such as finite elements are typically limited by inherent modeling assumptions as well as the prohibitive computational effort required for analysis. This necessitates the development of surrogate models that serve as a basis for predicting structural response. Deconvolution interferometry is a viable data-driven approach for such a task that uses single component sensor data to generate a set of impulse response functions for a structure of interest that constitutes the required surrogate model of the structure. The resulting surrogate model aids in both dynamic characterization as well as for accurate response prediction. However, it is limited to cases where motions in various degrees of freedom of a structure can be decoupled. This decoupling requires dense sensor deployment as well as prior knowledge about the structure's geometry. To overcome this limitation, multi-component deconvolution seismic interferometry approach can be used to develop a surrogate model for response prediction for cases with sparse sensor deployment and limited information about the structure of interest. The resulting model incorporates various sources of uncertainties namely measurement noise, effects of variations of temperature and humidity, and human activity induced vibrations by predicting a probabilistic structural response. For example, Figure 9 (right) shows the prediction perfromance from field monitoring data collected from structures with sparse sensor deployment in the Groningen region of the Netherlands (Figure 9, left) for a period of approximately 10 months on average.
D. Sen, J. Long, H. Sun, X. Campman, and O. Buyukozturk. Multi-component deconvolution interferometry for data-driven prediction of seismic structural response. Engineering Structures 2021, 241: 112405.
The structural health monitoring (SHM) process consists of various inter-related tasks such as sensor network management, energy optimization, information extraction, uncertainty quantification (UQ), and decision making. Accomplishment of each one of these tasks requires certain information and the objective of signal processing is to efficiently extract such information mainly from the raw signals provided by the sensor network. One of our proposed methods in this regards is the sparse generalized pencil-of-functions (SGPOF) that characterized a signal in terms of damped harmonic modes. In this method, we made use of sparsity-based regularization in order to exclude the spurious modes from the expansion of signals and hence, avoid energy leakage. Figure 10 shows a simple comparison between SGPOF and other common signal processing method.
Characterizing signals in complex plane in terms of complex poles and residues and the sparse property of this method brings a wide range of applications for SGPOF such as in seismic migration, solving electromagnetic interaction problems, etc. This method can be also used in SHM for tracking the changes in a system's complex poles as a basis for disturbance and damage detection. Figure 11 shows the variations of the poles of a cantilever beam as a result of damage. It also shows a comparison between the efficacies of SGPOF with auto-regressive (AR) coefficients for the use as damage sensitive features. This comparison was conducted based on based on the gradient boosting technique for feature selection.
The ultimate goals of SHM are to detect and localize damages, quantify the uncertainties associated with detection, and make appropriate decisions based on that quantification. In order to improve the localization accuracy and reduce the associated uncertainties we use pairwise graphical models to make inference on the state of structure around each sensor location. Figures 12 and 13 show two experiments and compare the result of node-wise damage detection techniques with the proposed one that employs graphical models. The first experiment was conducted on a model structure instrumented by MEMS accelerometers. In the second experiment, Figure 9, we used a video camera to extract the displacement field of a cantilever beam.
R. Mohammadi Ghazi, J. G. Chen and O. Buyukozturk. Pairwise graphical models for structural health monitoring with dense sensor arrays. Mechanical Systems and Signal Processing 2017; 93: 578-592.
Wireless sensor networks promise to deliver low cost, low power and massively distributed systems for structural health monitoring. These networks are comprised of many individual ‘nodes’, which each have sensing, computational and communication ability, and are battery powered. A key component of these systems, particularly when sampling rates are high, is the capability to process data within the network. This capability allows sensor data streams to be condensed at the sensor node level before wireless communication, often dramatically reducing the power used, and therefore extending the battery life of individual nodes in the network. Although progress has been made towards this vision, it remains a difficult task to develop and program ’smart’ wireless sensing applications. We are developing a system which allows data acquisition and computational tasks to be specified in Python, a high level programming language, and executed within the sensor network. Key features of this system include the ability to execute custom application code without firmware updates, to run multiple users’ requests concurrently and to conserve power through adjustable sleep settings.
To allow researchers and users to access to the wireless sensor network remotely, we are also developing a web-based interface. This interface helps researchers to develop and deploy their data acquisition and processing requests remotely, and to receive the results of the requests through their browser. In addition, we monitor and track the battery life, status, and health of the nodes in the network, helping to develop wireless sensing applications which are resource and environment aware.
Structural health monitoring (SHM) applications generally utilise high sampling rates, which low-power wireless protocols are not well equipped to handle. Smart sensing approaches can overcome this, by using the processing capability of the sensor nodes to reduce the volume of data prior to communication.
We propose a new computational framework for smart vibration sensing, and develop a sensor system which implements this framework. A critical advantage of this framework is its flexibility, allowing data processing logic to be remotely reconfigured almost instantaneously. We then extend this approach, developing a resource allocation algorithm which continually tracks network resources (battery life, computational power and communication bandwidth), and ensures that in-network computations utilize these resources optimally.
J. Long and O. Buyukozturk. A power optimised and re-programmable system for smart wireless vibration monitoring. Structural Control and Health Monitoring 2019; 27(2): e2468.
Energy harvesting wireless sensor networks are a promising solution for low cost, long lasting civil monitoring applications. But management of energy consumption is a critical concern to ensure these systems provide maximal utility. Many common civil engineering monitoring are fundamentally concerned with detecting and analysing infrequently occurring events, like high winds, earthquakes, or unusually heavy traffic loads . To conserve energy in these situations, a subset of nodes in the network can assume active duty, listening for events of interest, while the remaining nodes enter low power sleep mode to conserve battery. However, judicious planning of the sequence of active node assignments is needed to ensure that as many nodes as possible can be reached upon the detection of an event, and that the system maintains capability in times of low energy harvesting capabilities.
To better understand the behaviour of these networks, we use extensive computational simulations to model the change in battery levels and wireless connectivity under different environmental conditions. We then compare various strategies for duty cycling to evaluate their efficacy, including a novel deep reinforcement learning based approach, which uses the simulation data to learn to make decisions on powering on and off sensor nodes.
J. Long and O. Buyukozturk. Collaborative duty cycling strategies in energy harvesting sensor networks. Computer Aided Civil and Infrastructure Engineering, 2020; 35(6): 534-548.
The problem of automatic damage detection in civil structures is complex and requires a system that can interpret sensor data into meaningful information. We apply our recently developed switching Bayesian model for dependency analysis to the problems of damage detection, localization, and classification. The model relies on a state-space approach that accounts for noisy measurement processes and missing data. In addition, the model can infer statistical temporal dependency among measurement locations signifying the potential flow of information within the structure. By employing a fully Bayesian approach, we are able to characterize uncertainty in these variables via their posterior distribution and answer questions probabilistically, such as "What is the probability that damage has occurred?" and "Given that damage has occurred, what is the most likely damage scenario?".
Z. Dzunic, J. G. Chen, H. Mobahi, O. Buyukozturk and J. W. Fisher. A Bayesian state-space approach for damage detection and classification. Mechanical Systems and Signal Processing 2017; 96: 239-259.
Acoustic-laser vibrometry is a robust standoff NDT technique for composite materials, specifically implemented on FRP-reinforced concrete and FRP-steel composites. The technique exploits differing responses in intact and defective materials from external forcing. An acoustic wave excites a composite material and causes it to vibrate; defects such as voids and delaminations will vibrate excessively like a drum head, which are measured and identified by the laser vibrometer.
J. G. Chen, R. W. Haupt and O. Buyukozturk. Operational and Defect Parameters Concerning the Acoustic-Laser Vibrometry Method for FRP-reinforced Concrete. NDT&E International 2015; 71: 43-53.
The proposed approach intergrates inverse synthetic aperture radar (ISAR) measurements and a backprojection algorithm for codition assessment of glass fiber reinforced polymenr (GFRP)-retrofitted concrete structures. The method helps detect near-surface defects and delaminations located in the vicinity of GFRP-concrete interface.
T.-Y. Yu and O. Buyukozturk. A far-field airborne radar NDT technique for detecting debonding in GFRP�retrofitted concrete structures. NDT&E International 2008; 41: 10-24.
C. Rao, P. Ren, O. Buyukozturk, H. Sun, and Y. Liu. Encoding physics to learn reaction-diffusion processes. Nature Machine intelligence 2023, 5, 765�779.
D. Sen, J. Long, H. Sun, X. Campman, and O. Buyukozturk. Multi-component deconvolution interferometry for data-driven prediction of seismic structural response. Engineering Structures 2021, 241: 112405.
K. Kupwade-Patil, J.G. Chen, M. Uzun, D. Lau, M.L. Johnston, A. Zhou, D. Smit, and O. Buyukozturk. Corrosion assessment of ductile iron pipes using high-speed camera technique: Microstructural validation. NDT & E International 2020, 116: 102362.
E. Ghorbani, O. Buyukozturk and Y-J. Cha. Hybrid output-only structural system identification using random decrement and Kalman filter. Mechanical Systems and Signal Processing, 2020; 144: 106977.
R. Mohammadi-Ghazi, R.E. Welsch and O. Buyukozturk. Kernel dependence analysis and graph structure morphing for novelty detection with high-dimensional small size dataset. Mechanical Systems and Signal Processing 2020; 143: 106775
J. Long and O. Buyukozturk. Collaborative duty cycling strategies in energy harvesting sensor networks. Computer Aided Civil and Infrastructure Engineering, 2020; 35(6): 534-548.
J. Long and O. Buyukozturk. A power optimised and re-programmable system for smart wireless vibration monitoring. Structural Control and Health Monitoring 2019; 27(2): e2468.
M. Uzun, H. Sun, D. Smit, and O. Buyukozturk. Structural identification and damage detection using Bayesian inference and seismic interferometry. Structural Control and Health Monitoring 2019; DOI:10.1002/stc.2445
R. Zhang, Z. Chen, S. Chen, J. Zheng, O. Buyukozturk, and H. Sun. Deep long short-term memory networks for nonlinear structurl seismic response prediction. Computers and Structures 2019; 220: 55-68.
J. G. Chen, T. M. Adams, H. Sun, E. S. Bell and O. Buyukozturk. Camera-based vibration measurement of the Portsmouth, NH WWI Memorial Bridge. Journal of Structural Engineering, ASCE 2018; accepted 144(11); 04018207.
H. Sun, J. Al-Qazweeni, J. Parol, H. Kamal, Z. Chen and O. Buyukozturk. Computational modeling of a unique tower in Kuwait for structural health monitoring: numerical investigations. Structural Control & Health Monitoring 2019; 26(3): e2317
R. Mohammadi-Ghazi, Y. M. Marzouk and O. Buyukozturk. Conditional classifiers and boosted conditional Gaussian mixture model for novelty detection. Pattern Recognition 2018; 81: 601-614.
H. Sun and O. Buyukozturk. The MIT Green Building benchmark problem for structural health monitoring of tall buildings. Structural Control and Health Monitoring 2018; 25(3): e2115.
Y-J. Cha, W. Choi, G. Suh, S. Mahmoudkhani and O. Buyukozturk. Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types. Computer-Aided Civil and Infrastructure Engineering 2017; DOI:10.1111/mice.12334.
A. Mordret, H. Sun, G. A. Prieto, M. N. Toksoz and O. Buyukozturk. Continuous monitoring of high-rise buildings using seismic interferometry. Bulletin of the Seismological Society of America 2017; 107(6): 2759-2773.
J. G. Chen and O. Buyukozturk. A symmetry measure for detecting changes in mode shapes. Journal of Sound and Vibration 2017; 408: 123-137.
N. Wadhwa, J. G. Chen, J. B. Sellon, D. Wei, M. Rubinstein, R. Ghaffari, D. M. Freeman, O. Buyukozturk, P. Wang, S. Sun, S. H. Kang, K. Bertoldi, F. Durand and W. T. Freeman. Motion microscopy for visualizing and quantifying small motions. Proceedings of the National Academy of Sciences 2017; 201703715.
Z. Dzunic, J. G. Chen, H. Mobahi, O. Buyukozturk and J. W. Fisher. A Bayesian state-space approach for damage detection and classification. Mechanical Systems and Signal Processing 2017; 96: 239-259.
R. Mohammadi Ghazi, J. G. Chen and O. Buyukozturk. Pairwise graphical models for structural health monitoring with dense sensor arrays. Mechanical Systems and Signal Processing 2017; 93: 578-592.
Y-J. Cha, W. Choi and O. Buyukozturk. Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering 2017; 32(5): 361-378.
J. Long and O. Buyukozturk. Decentralised One Class Kernel Classification Based Damage Detection and Localisation. Structural Control and Health Monitoring 2017; 24(6): e1930.
A. Davis, K.L. Bouman, J.G. Chen, M. Rubinstein, O. Buyukozturk, F. Durand and W.T. Freeman. Visual Vibrometry: Estimating Material Properties from Small Motions in Video. IEEE Transactions on Pattern Analysis and Machines Intelligence 2017; 39(4): 732-745.
Y-J. Cha, J. G. Chen and O. Buyukozturk. Output-only computer vision based damage detection using phase-based optical flow and unscented Kalman filters. Engineering Structures 2017; 132, 300-313. <
H. Sun, A. Mordret, G. A. Prieto, N. Toksoz and O. Buyukozturk. Bayesian characterization of buildings using seismic interferometry on ambient vibrations. Mechanical Systems and Signal Processing 2017; 85: 468-486.
G. Yan, H. Sun and O. Buyukozturk. Impact load identification for composite structures using Bayesian regularization and unscented Kalman filter. Structural Control and Health Monitoring 2016; 24(5): e1910.
J. G. Chen, A. Davis, N. Wadhwa, F. Durand, W. T. Freeman and O. Buyukozturk. Video camera-based vibration measurement for civil infrastructure applications. ASCE Journal of Infrastructure Systems 2016; in press. DOI: 10.1061/(ASCE)IS.1943-555X.0000348
N. Wadhwa, H-Y. Wu, A. Davis, M. Rubinstein, E. Shih, G. J. Mysore, J. G. Chen, O. Buyukozturk, J. V. Guttag, W. T. Freeman and F. Durand. Eulerian video magnification and analysis. Communications of ACM 2016; 60(1): 87-95.
Y-J. Cha, P. Trocha and O. Buyukozturk. Field measurement based system identification and dynamic response prediction of a unique MIT building. Sensors 2016; 16(7): 1016.
H. Sun and O. Buyukozturk. Probabilistic updating of building models using incomplete modal data. Mechanical Systems and Signal Processing 2016; 75: 27-40.
R. Mohammadi Ghazi and O. Buyukozturk. Damage detection with small data set using energy-based nonlinear features. Structural Control and Health Monitoring 2016; 23(2): 333-348.
H. Sun and O. Buyukozturk. Optimal sensor placement in structural health monitoring using discrete optimization. Smart Materials and Structures 2015; 24(12): 125034.
H. Sun and O. Buyukozturk. Identification of traffic-induced nodal excitation of truss bridges through heterogeneous data fusion. Smart Materials and Structures 2015; 24(7): 075032.
Y-J. Cha and O. Buyukozturk. Structural Damage Detection Using Modal Strain Energy and Hybrid Multiobjective Optimization. Computer-Aided Civil and Infrastructure Engineering 2015; 30(5): 347-358. <
J. G. Chen, N. Wadhwa, Y.-J. Cha, F. Durand, W. T. Freeman and O. Buyukozturk. Modal Identification of Simple Structures with High-Speed Video using Motion Magnification. Journal of Sound and Vibration 2015; 345: 58-71.
J. G. Chen, R. W. Haupt and O. Buyukozturk. Operational and Defect Parameters Concerning the Acoustic-Laser Vibrometry Method for FRP-reinforced Concrete. NDT&E International 2015; 71: 43-53.
T.-Y. Yu and O. Buyukozturk. A far-field airborne radar NDT technique for detecting debonding in GFRP retrofitted concrete structures. NDT&E International 2008; 41: 10-24.
Back to top