SENSING and MONITORING

Video Camera-based Structural Monitoring - Motion Magnification

The Motion Magnification algorithm can magnify small motions in videos in specific bands of frequencies; for structures if you choose the frequencies to coincide with resonant frequencies you can create videos of the approximate or operational mode shapes. Expanding on this, using phase based optical flow, quantitative displacements can be measured from videos and mode shapes can be identified.

 

Illustrations of motion magnification


Currently, in pseudo real-time we can take a video of a small structure, identify and measure its operational mode shapes, and generate motion magnified video at the identified resonant frequencies. Vibrational inspection of small structures and machinery is currently being developed. For measurements of distant civil infrastructure, issues such as atmospheric turbulence and camera platform motion will be investigated.

 

Video of a motion magnified pipe. From left to right: Source, 480 Hz, 1200 Hz, 2400 Hz.


J. G. Chen, N. Wadhwa, Y.-J. Cha, F. Durand, W. T. Freeman and O. Buyukozturk. Modal Identification of Simple Structures with High-Speed Video using Motion Magnification. Journal of Sound and Vibration 2015; 345: 58-71. [highlighted by MIT News]

Video of WW I Memorial Bridge in Portsmouth, NH. From left to right: Source, bending mode and twisting mode.


J. G. Chen, T. M. Adams, H. Sun, E. S. Bell and O. Buyukozturk. Camera-based vibration measurement of the Portsmouth, NH WWI Memorial Bridge. Journal of Structural Engineering, ASCE 2018; 144(11); 04018207.

Structural Health Monitoring of Tall Buildings

Vibration-based structural health monitoring (SHM) provides a primary tool for evaluating structural condition, integrity and reliability as well as for assessing potential risks throughout the lifecycle of structures. The objective of this research is to propose an integrated sensing and decision-making system for SHM of tall buildings (see Figure 1). This research addresses challenges related to computational modeling, vibration sensor optimal deployment, model updating and data analytics including mining, identification, uncertainty quantification, etc. A continuous monitoring system can build a temporal database for the building and detect possible damage due to structural deterioration or extreme events at an early stage.

 

Figure 1: Structural Health Monitoring of Tall Buildings


Computational modeling

Computational modeling, in addition to data analytics, plays an important role in structural health monitoring (SHM). A high fidelity computational model based on teh design and construction information provides important dynamic characteristics of a structure. Additionally, it may be used for devising sensor placement strategies, and based on acquired field measurements it can be updated for SHM puposes such as damage detection, response prediction and reliability assessment. The following figures show the finite element model developed for the world's tallest sculptured RC tower, Al Hamra building in Kuwait City, Kuwait.

 

 

Figure: From left to right, Modes 1 to 3 of the Al Hamra Tower, response of the tower subjected to scaled Chi-Chi ground motion.

H. Sun, J. Al-Qazweeni, J. Parol, H. Kamal, Z. Chen and O. Buyukozturk. Computational modeling of a unique tower in Kuwait for structural health monitoring: numerical investigations. Structural Control & Health Monitoring 2019; 26(3): e2317

 

Optimal sensor placement

The objective of optimal sensor placement (OSP) is to obtain a sensor layout that gives as much information of the dynamic system as possible in SHM. The process of OSP can be formulated as a discrete minimization (or maximization) problem with the sensor locations as the design variables, conditional on the constraint of a given sensor number. A finite element model-based optimal sensor placement algorithm is investigated for building instrumentation and has been successfully validated on the Green Building (83.47 m) at MIT shown in Figure 2.

 

Figure 2: Optimal sensor placement for building monitoring


Figure: Results from Optimal sensor placement algorithm versus actual monitoring system deployment on Al Hamra Tower, Kuwait City, Kuwait


H. Sun and O. Buyukozturk. Optimal sensor placement in structural health monitoring using discrete optimization. Smart Materials and Structures 2015; 24(12): 125034.

 

Temporal building characterization through seismic interferometry

Continuous monitoring of buildings provides a crucial alternative to assess its health condition as well as evaluate its safety throughout the whole service life. The vibration of a building is related to the excitation, the soil-structure interaction, and the building mechanical properties. Separating the building response from the excitation and the soil-structure interaction using vibrational data yields information of the intrinsic characteristics of the building. A deconvolution-based a seismic interferometric method is proposed to extract the shear waves propagating in the building (see Figure 3). Intrinsic physical properties such as wave velocity, mode shapes and damping ratios can be continuously extracted if the building is monitored temporally (see Figure 4). The identified building properties can then be used for structural condition evaluation, model updating, damage detection, etc.

 

Figure 3: Shear waves propagating in the MIT Green Building extracted through seismic interferometry


Figure 4: Mode shape extraction and damping identification of building


H. Sun, A. Mordret, G. A. Prieto, N. Toksoz and O. Buyukozturk. Bayesian characterization of buildings using seismic interferometry on ambient vibrations. Mechanical Systems and Signal Processing 2017; 85: 468-486. [Highlighted on MIT News]

 

Uncertainty quantification of building models

Quantification of the uncertain parameters of a building model is important in SHM since it is related to assessing the health condition, evaluating the integrity, and estimating the capacity to carry loads and risk. The objective of this study is to establish a numerical model, calibrated against field records, for building response prediction subjected to potential extreme events and for damage detection in future operations of the building. A hierarchical Bayesian inference approach is investigated to quantify the uncertainties of the model parameters. For example, the distributions of storey stiffnesses can be continuously updated over time and used for possible building damage detection after an extreme event such as earthquake (see Figure 5).

 

Figure 5: Uncertainty quantification of building model parameters


Damage and Disturbance Detection

The structural health monitoring (SHM) process consists of various inter-related tasks such as sensor network management, energy optimization, information extraction, uncertainty quantification (UQ), and decision making. Accomplishment of each one of these tasks requires certain information and the objective of signal processing is to efficiently extract such information mainly from the raw signals provided by the sensor network. One of our proposed methods in this regards is the sparse generalized pencil-of-functions (SGPOF) that characterized a signal in terms of damped harmonic modes. In this method, we made use of sparsity-based regularization in order to exclude the spurious modes from the expansion of signals and hence, avoid energy leakage. Figure 6 shows a simple comparison between SGPOF and other common signal processing method.

 

Figure 6: Comparing SGPOF with Fourier transform for modal frequency estimation and detection of discontinuities of signal


Characterizing signals in complex plane in terms of complex poles and residues and the sparse property of this method brings a wide range of applications for SGPOF such as in seismic migration, solving electromagnetic interaction problems, etc. This method can be also used in SHM for tracking the changes in a system's complex poles as a basis for disturbance and damage detection. Figure 3 shows the variations of the poles of a cantilever beam as a result of damage. It also shows a comparison between the efficacies of SGPOF with auto-regressive (AR) coefficients for the use as damage sensitive features. This comparison was conducted based on based on the gradient boosting technique for feature selection.

 

Figure 7: Comparing SGPOF with AR for extracting damage sensitive features


The ultimate goals of SHM are to detect and localize damages, quantify the uncertainties associated with detection, and make appropriate decisions based on that quantification. In order to improve the localization accuracy and reduce the associated uncertainties we use pairwise graphical models to make inference on the state of structure around each sensor location. Figure 8 and 9 show two experiments and compare the result of node-wise damage detection techniques with the proposed one that employs graphical models. The first experiment was conducted on a model structure instrumented by MEMS accelerometers. In the second experiment, Figure 9, we used a video camera to extract the displacement field of a cantilever beam.


Figure 8: damage localization using the proposed method with graphical models and compared with SVM


Figure 9: Perfect damage localization using the proposed techniques with graphical models


R. Mohammadi Ghazi, J. G. Chen and O. Buyukozturk. Pairwise graphical models for structural health monitoring with dense sensor arrays. Mechanical Systems and Signal Processing 2017; 93: 578-592.

 

Wireless Sensor Networks

Wireless sensor networks promise to deliver low cost, low power and massively distributed systems for structural health monitoring. These networks are comprised of many individual ‘nodes’, which each have sensing, computational and communication ability, and are battery powered. A key component of these systems, particularly when sampling rates are high, is the capability to process data within the network. This capability allows sensor data streams to be condensed at the sensor node level before wireless communication, often dramatically reducing the power used, and therefore extending the battery life of individual nodes in the network. Although progress has been made towards this vision, it remains a difficult task to develop and program ’smart’ wireless sensing applications. We are developing a system which allows data acquisition and computational tasks to be specified in Python, a high level programming language, and executed within the sensor network. Key features of this system include the ability to execute custom application code without firmware updates, to run multiple users’ requests concurrently and to conserve power through adjustable sleep settings.

 

Figure 10: System schematic - Remote programming and management of wireless sensor networks


To allow researchers and users to access to the wireless sensor network remotely, we are also developing a web-based interface. This interface helps researchers to develop and deploy their data acquisition and processing requests remotely, and to receive the results of the requests through their browser. In addition, we monitor and track the battery life, status, and health of the nodes in the network, helping to develop wireless sensing applications which are resource and environment aware.

 

Figure 11: Snapshot of web interface for sensor programming and management


Reprogrammable distributed smart sensing networks

Structural health monitoring (SHM) applications generally utilise high sampling rates, which low-power wireless protocols are not well equipped to handle. Smart sensing approaches can overcome this, by using the processing capability of the sensor nodes to reduce the volume of data prior to communication.
We propose a new computational framework for smart vibration sensing, and develop a sensor system which implements this framework. A critical advantage of this framework is its flexibility, allowing data processing logic to be remotely reconfigured almost instantaneously. We then extend this approach, developing a resource allocation algorithm which continually tracks network resources (battery life, computational power and communication bandwidth), and ensures that in-network computations utilize these resources optimally.

Figure: Sensor node hardware including low power wireless radio, computational core and MEMS accelerometer


Figure: Schematic of the Beeview smart wireless sensing architecture, illustrating how individual sensing tasks are passed from the client application, through the web server where they are optimized based on the network configuration and specified sensing algorithm, and finally to the deployed sensor network where data is acquired, processed, and returned.


Figure: Implementation of the decentralised frequency domain decomposition algorithm, illustrating how the network is divided into overlapping neighborhoods, allowing parallel estimation of modeshapes for each neighborhood which are ultimately combined to return a noisy estimate of the full network modeshape.


J. Long and O. Buyukozturk. A power optimised and re-programmable system for smart wireless vibration monitoring. Structural Control and Health Monitoring 2019; 27(2): e2468.

 

Collaborative duty cycling in energy harvesting sensor networks

Energy harvesting wireless sensor networks are a promising solution for low cost, long lasting civil monitoring applications. But management of energy consumption is a critical concern to ensure these systems provide maximal utility. Many common civil engineering monitoring are fundamentally concerned with detecting and analysing infrequently occurring events, like high winds, earthquakes, or unusually heavy traffic loads . To conserve energy in these situations, a subset of nodes in the network can assume active duty, listening for events of interest, while the remaining nodes enter low power sleep mode to conserve battery. However, judicious planning of the sequence of active node assignments is needed to ensure that as many nodes as possible can be reached upon the detection of an event, and that the system maintains capability in times of low energy harvesting capabilities.
To better understand the behaviour of these networks, we use extensive computational simulations to model the change in battery levels and wireless connectivity under different environmental conditions. We then compare various strategies for duty cycling to evaluate their efficacy, including a novel deep reinforcement learning based approach, which uses the simulation data to learn to make decisions on powering on and off sensor nodes.

Figure: Schematic showing how the centralized power manager tracks and controls the state of the network at discrete time intervals. Example simulation of solar energy availability and wireless connectivity on right.


J. Long and O. Buyukozturk. Collaborative duty cycling strategies in energy harvesting sensor networks. Computer Aided Civil and Infrastructure Engineering, 2020; 35(6): 534-548.

 

Inference in Graphical Models

The problem of automatic damage detection in civil structures is complex and requires a system that can interpret sensor data into meaningful information. We apply our recently developed switching Bayesian model for dependency analysis to the problems of damage detection, localization, and classification. The model relies on a state-space approach that accounts for noisy measurement processes and missing data. In addition, the model can infer statistical temporal dependency among measurement locations signifying the potential flow of information within the structure. By employing a fully Bayesian approach, we are able to characterize uncertainty in these variables via their posterior distribution and answer questions probabilistically, such as "What is the probability that damage has occurred?" and "Given that damage has occurred, what is the most likely damage scenario?".

 

Figure 12: State-space switching interaction model (SSIM)


Z. Dzunic, J. G. Chen, H. Mobahi, O. Buyukozturk and J. W. Fisher. A Bayesian state-space approach for damage detection and classification. Mechanical Systems and Signal Processing 2017; 96: 239-259.

 

Deep learning-based damage detection

The structural health monitoring research community has recently started the application of image processing techniques for vision-based monitoring of infrastructure systems. Such techniques typically involve extensive image manipulation and feature extraction for reliable damage detection. However, variablity of real-world scenarios, like lighting and shadow changes, hinders the applicability of such algorithms for field monitoring. To overcome these challenges, a deep convolutional neural network (CNN)-based framework employed for damage detection instead, which has demonstrated its efficacy in dealing with real-world scenarios.

Figure 13: A flowchart showing the deep CNN-based algorithm for surface crack detection in concrete structures.


Figure 14: Detection of corrosion and delamination using a region-based CNN.


Y-J. Cha, W. Choi and O. Buyukozturk. Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering 2017; 32(5): 361-378.

 

Acoustic-Laser Vibrometry

Acoustic-laser vibrometry is a robust standoff NDT technique for composite materials, specifically implemented on FRP-reinforced concrete and FRP-steel composites. The technique exploits differing responses in intact and defective materials from external forcing. An acoustic wave excites a composite material and causes it to vibrate; defects such as voids and delaminations will vibrate excessively like a drum head, which are measured and identified by the laser vibrometer.

 

Figure 15: Acoustic-laser vibrometry for NDE


Figure 16: Operational mode shapes of a defect in FRP-retrofitted concrete as measured by acoustic-laser vibrometry


J. G. Chen, R. W. Haupt and O. Buyukozturk. Operational and Defect Parameters Concerning the Acoustic-Laser Vibrometry Method for FRP-reinforced Concrete. NDT&E International 2015; 71: 43-53.

 

Far-field airborne radar NDT technique for detetcting debonding in GFRP retrofitted concrete structures

The proposed approach intergrates inverse synthetic aperture radar (ISAR) measurements and a backprojection algorithm for codition assessment of glass fiber reinforced polymenr (GFRP)-retrofitted concrete structures. The method helps detect near-surface defects and delaminations located in the vicinity of GFRP-concrete interface.

 

Figure: (top) Damages studied in an experimental setting (left) Results for the undamaged case (middle) Results for damaged case 1 (right) Results for damaged case 2


T.-Y. Yu and O. Buyukozturk. A far-field airborne radar NDT technique for detecting debonding in GFRP–retrofitted concrete structures. NDT&E International 2008; 41: 10-24.

 

Recent Related Publications

R. Mohammadi-Ghazi, R.E. Welsch and O. Buyukozturk. Kernel dependence analysis and graph structure morphing for novelty detection with high-dimensional small size dataset. Mechanical Systems and Signal Processing 2020; 143: 106775

J. Long and O. Buyukozturk. Collaborative duty cycling strategies in energy harvesting sensor networks. Computer Aided Civil and Infrastructure Engineering, 2020; 35(6): 534-548.

J. Long and O. Buyukozturk. A power optimised and re-programmable system for smart wireless vibration monitoring. Structural Control and Health Monitoring 2019; 27(2): e2468.

M. Uzun, H. Sun, D. Smit, and O. Buyukozturk. Structural identification and damage detection using Bayesian inference and seismic interferometry. Structural Control and Health Monitoring 2019; DOI:10.1002/stc.2445

R. Zhang, Z. Chen, S. Chen, J. Zheng, O. Buyukozturk, and H. Sun. Deep long short-term memory networks for nonlinear structurl seismic response prediction. Computers and Structures 2019; 220: 55-68.

J. G. Chen, T. M. Adams, H. Sun, E. S. Bell and O. Buyukozturk. Camera-based vibration measurement of the Portsmouth, NH WWI Memorial Bridge. Journal of Structural Engineering, ASCE 2018; accepted 144(11); 04018207.

H. Sun, J. Al-Qazweeni, J. Parol, H. Kamal, Z. Chen and O. Buyukozturk. Computational modeling of a unique tower in Kuwait for structural health monitoring: numerical investigations. Structural Control & Health Monitoring 2019; 26(3): e2317

R. Mohammadi-Ghazi, Y. M. Marzouk and O. Buyukozturk. Conditional classifiers and boosted conditional Gaussian mixture model for novelty detection. Pattern Recognition 2018; 81: 601-614.

H. Sun and O. Buyukozturk. The MIT Green Building benchmark problem for structural health monitoring of tall buildings. Structural Control and Health Monitoring 2018; 25(3): e2115.

Y-J. Cha, W. Choi, G. Suh, S. Mahmoudkhani and O. Buyukozturk. Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types. Computer-Aided Civil and Infrastructure Engineering 2017; DOI:10.1111/mice.12334.

A. Mordret, H. Sun, G. A. Prieto, M. N. Toksoz and O. Buyukozturk. Continuous monitoring of high-rise buildings using seismic interferometry. Bulletin of the Seismological Society of America 2017; 107(6): 2759-2773.

J. G. Chen and O. Buyukozturk. A symmetry measure for detecting changes in mode shapes. Journal of Sound and Vibration 2017; 408: 123-137.

N. Wadhwa, J. G. Chen, J. B. Sellon, D. Wei, M. Rubinstein, R. Ghaffari, D. M. Freeman, O. Buyukozturk, P. Wang, S. Sun, S. H. Kang, K. Bertoldi, F. Durand and W. T. Freeman. Motion microscopy for visualizing and quantifying small motions. Proceedings of the National Academy of Sciences 2017; 201703715.

Z. Dzunic, J. G. Chen, H. Mobahi, O. Buyukozturk and J. W. Fisher. A Bayesian state-space approach for damage detection and classification. Mechanical Systems and Signal Processing 2017; 96: 239-259.

R. Mohammadi Ghazi, J. G. Chen and O. Buyukozturk. Pairwise graphical models for structural health monitoring with dense sensor arrays. Mechanical Systems and Signal Processing 2017; 93: 578-592.

Y-J. Cha, W. Choi and O. Buyukozturk. Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering 2017; 32(5): 361-378.

J. Long and O. Buyukozturk. Decentralised One Class Kernel Classification Based Damage Detection and Localisation. Structural Control and Health Monitoring 2017; 24(6): e1930.

A. Davis, K.L. Bouman, J.G. Chen, M. Rubinstein, O. Buyukozturk, F. Durand and W.T. Freeman. Visual Vibrometry: Estimating Material Properties from Small Motions in Video. IEEE Transactions on Pattern Analysis and Machines Intelligence 2017; 39(4): 732-745.

Y-J. Cha, J. G. Chen and O. Buyukozturk. Output-only computer vision based damage detection using phase-based optical flow and unscented Kalman filters. Engineering Structures 2017; 132, 300-313. <

H. Sun, A. Mordret, G. A. Prieto, N. Toksoz and O. Buyukozturk. Bayesian characterization of buildings using seismic interferometry on ambient vibrations. Mechanical Systems and Signal Processing 2017; 85: 468-486.

G. Yan, H. Sun and O. Buyukozturk. Impact load identification for composite structures using Bayesian regularization and unscented Kalman filter. Structural Control and Health Monitoring 2016; 24(5): e1910.

J. G. Chen, A. Davis, N. Wadhwa, F. Durand, W. T. Freeman and O. Buyukozturk. Video camera-based vibration measurement for civil infrastructure applications. ASCE Journal of Infrastructure Systems 2016; in press. DOI: 10.1061/(ASCE)IS.1943-555X.0000348

N. Wadhwa, H-Y. Wu, A. Davis, M. Rubinstein, E. Shih, G. J. Mysore, J. G. Chen, O. Buyukozturk, J. V. Guttag, W. T. Freeman and F. Durand. Eulerian video magnification and analysis. Communications of ACM 2016; 60(1): 87-95.

Y-J. Cha, P. Trocha and O. Buyukozturk. Field measurement based system identification and dynamic response prediction of a unique MIT building. Sensors 2016; 16(7): 1016.

H. Sun and O. Buyukozturk. Probabilistic updating of building models using incomplete modal data. Mechanical Systems and Signal Processing 2016; 75: 27-40.

R. Mohammadi Ghazi and O. Buyukozturk. Damage detection with small data set using energy-based nonlinear features. Structural Control and Health Monitoring 2016; 23(2): 333-348.

H. Sun and O. Buyukozturk. Optimal sensor placement in structural health monitoring using discrete optimization. Smart Materials and Structures 2015; 24(12): 125034.

H. Sun and O. Buyukozturk. Identification of traffic-induced nodal excitation of truss bridges through heterogeneous data fusion. Smart Materials and Structures 2015; 24(7): 075032.

Y-J. Cha and O. Buyukozturk. Structural Damage Detection Using Modal Strain Energy and Hybrid Multiobjective Optimization. Computer-Aided Civil and Infrastructure Engineering 2015; 30(5): 347-358. <

J. G. Chen, N. Wadhwa, Y.-J. Cha, F. Durand, W. T. Freeman and O. Buyukozturk. Modal Identification of Simple Structures with High-Speed Video using Motion Magnification. Journal of Sound and Vibration 2015; 345: 58-71.

J. G. Chen, R. W. Haupt and O. Buyukozturk. Operational and Defect Parameters Concerning the Acoustic-Laser Vibrometry Method for FRP-reinforced Concrete. NDT&E International 2015; 71: 43-53.

T.-Y. Yu and O. Buyukozturk. A far-field airborne radar NDT technique for detecting debonding in GFRP retrofitted concrete structures. NDT&E International 2008; 41: 10-24.

 

Back to top